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SUMMARY 



The Title IV student financial assistance programs provided $15.4 billion in loans, grants, 
and work-study support to postsecondaiy students during the 1.988-89 award year. To 
evaluate and improve the qualit>' of the delivery of Title IV financial aid, the U.S. 
Department of Education contracted with Price Waterhouse to conduct the Integrated 
Quality Control Measurement Project (IQCMP). IQCMP measured the quality of 
awards distributed during the 1988-89 award year under the three major Title IV 
programs: the Pell Grant program, the Campus-Based programs (Supplemental 
Educational Opportunity Grant, College Work-Study, and Perkins Loan), and the 
Stafford Loan program. 

IQCMP relied on a nationwide: sample of Title IV recipients. For each sampled 
recipient, documentation was collected from educational institutions, students and 
parents, the Internal Revenue Service, and other sources to support or refute the data 
used to calculate the financial aid award. Error in the delivery of financial aid was then 
calculated based on the difference between the award acmally distributed and the award 
that would be calculated based on the best available data. 



IQCMP found that approximately 10.9 percent of the $15,4 billion dollars awarded 
through the Title IV programs was awarded in enor. This error was decomposed by 
program and source of error. Certain items on the financial aid application and certain 
steps in the award calculation process were found to be particularly error-prone, and 
alternative approaches to reducing these errors were identified. 

Integrated Verification, the primary procedure for verifying student application data, was 
found to be poorly targeted to students with error. While dependent smdents and 
students with large family income were much more likely than other students to have an 

ERIC 12 



IQCMP Executivg Summafv 



enor in their award, these error-prone groups were generally not selected for verification 
more often than other student groups. 

Although error rates have decreased over time, the complexity of the current delivery 
system ensures that, imless major changes are made, error will remain substantial. Two 
such changes are simplification of the deUvery system and placing a larger responsibility 
for quality control on educational institutions. We found evidence that both types of 
changes could be successful. First, we found that it was possible to simplify the financial 
aid application without introducing significant changes to the distribution of financial aid 
awards. Secondly, we found that many educational institutions currently have a 
significant commitment to quality. However, such major changes to the delivery system 
will require legislative action. We recommend that the Department of Education review 
its goals and determine whether such changes are warranted. 
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L INlRODUCnON 

A» The Integrated Quality Control Measurement Project 

1. Background and Objectives 

The Integrated Quality Control Measurement Project (IQCMP) is the most recent in a 
series of quality control studies conducted by the U.S. Department of Education (ED) to 
evaluate the quality of the delivery of Title IV smdent financial assistance. IQCMP 
evaluated the following Title IV programs for the 1988-89 academic year: 

• The Pell Grant Program, 

• The Campus-Based Programs (Supplemental Educational Opportunity 
Grant, College Work-Smdy, and Perkins Loan), and 

• The Stafford Loan Program (formerly the Guaranteed Student Loan 
Program). 

These Title IV programs distributed over $15.6 billion in grants, loans, and work-study to 
over 5.5 million students attending postsecondary institutions during the 1988-89 award 
year. Exhibit M presents a breakdown of dollars and awards by program. Over 8,000 
postsecondary institutions and more than 10,000 lending institutions and guarantee 
agencies participate in the delivery system. 

The study was sponsored by the Statistical Analysis Branch of the Division of Quality 
Assurance in the Debt Collection and Management Assistance Service of the Office of 
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TITLE IV AID AVAILABLE TO STUDENTS 

Award Year 1988-89 




Pell Grants 



4,863 



3,302,000 



1,473 



Campus-Based 
Woik Study 

SEOG 

Perkins Loan 
Total Campus Based 



780 
442 
884 



2,106 



835,000 
633,000 

826,000 
2,294,000 



934 
698 

1,070 
918 



Stafford Loans 



Total 



8,431 



15,400 



3,234,000 



8,830,000 



2,607 



1,744 



Data from 1991 budget request provided by the Department of Education 
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Postsecondaiy Edacation, U.S. Department of EducatioiL The study was conducted by 
Price Waterhousc in afff^ation with The Gallup Organization and Pelavin Associates, 
Inc. 

This document presents an overview of IQCMP, a summary of the findings on error in 
the delivery of Title IV stodent financial aid, and proposed corrective actions that are 
aimed at reducing error in future years. Detailed discussions on these topics can be 
found in other reports prepared as part of this project, including; 

• IQCMP Error Definitions - detailed definitions of each type of error 
examined during the project 

• TOT Pilot Analysis ~ an analysis of the Instimtional Quality Control Pilot 
Project 

• TQCMP Findinp and Corrective Actions - a detailed discussion of 
findings on error in the delivery of Title IV smdent financial aid and 
corrective actions aimed at reducing error in future years. 

2. IQCMP Methodology 

The IQCMP relied on a nationwide sample of 3,310 students attending 350 
postsecondaiy institutions to make inferences at the national level about the quality of 
the award process for the various Title IV financial aid programs. For each sampled 
smdent and instimtion, data was collected from the following sources: 

• Financial aid administrators at each institution were interviewed about the 
instimtion's financial aid policies and procedures. 
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• Infonnatioii from the school's finanffial aid records was abstracted for each 
of the san^)led students. Inconsistencies in the institution's financial aid 
records were discussed with the financial aid administrator to obtain an 
explanation or to confirm an error. 

• Interviews were conducted with sampled students and their parents. 
During these interviews, sludents and parents were requested to show 
documents verifying the data reported on the smdent's financial aid 
application. 

• Additional information from sources such as the IRS, financial instimtions, 
and property tax assessors was also collected to confirm or refute data 
reported on the student's financial aid application. 

Information from all sources was combined into a comprehensive database. This 
database was used to calculate Ijest** award values (based on the most reliable 
information collected from among the various sources) and to make projections of 
differences between best awards and acmal awards received. 



B. Definitions of Error in IQCMP 



1. IQCMP Error Measurement Philosophy 

The objective of IQCMP is to measure the xjualitt oi the financial aid delivery system 
and to identify steps that can be taken to improve quality. The term "error" as defined 
in IQCMP therefore refers to a deficiency in quality, which is a much broader definition 
of error than that included in the Title IV regulations. 



In general terms, error is defined to be the extent to which the delivery of Title IV aid 
differs from that intended under the Title IV regulations. This definition includes the 

4 
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following types of error 

• Institutional Uabilitv Error - A limited group of errors defined in the Title 
IV regulations. Institutions who commit liability errors must reimburse the 
Federal government for the amount of the error. 

• Institutional faihire to follow the Title IV regulations - A failure on the 
part of the institution to follow the Titie IV regulations, even though that 
failure is not a liability error. These errors include incorrect calculation of 
the financial aid award, failure to ensure smdent eUgibility, and failure to 
maintain documentation required imder the regulations. 

• Institutional failure to follow its own award policies - A failure by the 
institotion to follow its own poHcies for Title IV aid, even though tiiat 
failure may not violate tiie Titie IV regulations. 

• Intentional student reporting errors - An intentional failure by the student 
to report the correct data on his or her financial aid application. Under 
tiie Titie IV regulations, students are held accountable for these errors and 
may be subject to fines and/or imprisoimient. 

• Unintentional student reporting errors - An unintentional failure by the 
student to report the correct data on his or her financial application. 
Under the Titie IV regulations, the student's aid award should be adjusted 
if unintentional errors are found during verification of the student aid 
application. However, the student is not subject to fines or imprisonment. 

• Errors in estimates and projections - The award calculation relies in part 
on estimates (e.g., estimates of the value of a student's home, other real 
estate investments, etc.) and projections (e.g., the household size for the 
upcoming academic year, outside financial assistance for the upcoming 
academic year, etc.). Incorrect estimates and projections, if made in good 
faith, do not represent errors under the Titie IV regulations. IQCMP 
inchides incorrect estimates and projections as errors in order to evaluate 
the extent to which financial aid awards are consistent with the intent of 
the Titie IV program. 
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2. Definition of Error Measures 

Three* measures of error are used to describe the quaUty of the Title IV programs; 

• pftll Payment Error - error in payments from the Pell program. 

• raTnpiK..Bafsed Negri Error - the extent to which the calculated student 
finandai need differs from the best financial need. Under the Title IV 
regulations, the Campus-Based award may not exceed financial need. Due 
to limited funding, however, Campus-Based awards frequenrty do not fully 
meet the student's financial need. 

• Rtnffnrri T/>an Oy ftr^rtifiratinn Error - the extem to which the calailated 
Stafford Loan certification amount (the maximum loan amount avaUable to 
the student) exceeds the best certification amount. 

Each of the tiiree types of error listed above is broken down according to three different 
error sources: 

• Student Error - error resulting from inaccurate data reported by the 
student recipient 

• I pgtitntinnal Error - error resulting from inaccurate processing by the 
institutions or from improperly applying instimtion packaging guidelines. 

• Overall Error - error resulting from eitiier student or institutional error as 
defined above. 

It should be noted that smdem error and institutional error do not necessarily sum to the 
overaU error because errors made by students and institutions do not always have an 
additive effect on the overall award. AU errors presented in this report are calculated 



• Pr>u, o.h*r tvMi Of enor rCorapo«ite Enor. Csao^BiMd DiitributioMl Error. Campu^Bued Awiidi in Exce« of Need, and 
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with a $50 tolerance; that is, a best award within plus or minus $50 of the actual award is 
treated as zero error. 

This Executive Summary presents results for absolute^ error, which is the sum of 
overawards and underawards (where both overawards and underawards are considered to 
be positive). Due to limitations in available data, only overcertifications are included for 
Stafford Loan Certification Error. 



' Bttakoutt of ovenwird enor and undenwird error are dtscusaed in lOCMP Findints and Corrective Action?. 
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IL ERROR IN THE TITLE IV FINANCIAL AID PROGRAMS 

Exhibit n-l presents a summaiy of absolute error for each of the three Title IV 
programs. This chart shows the overall, student, and institutiona: percent of dollars in 
error and percent of recipients with error as well as the total dollars and the total 
number of awards for each of the programs. Note that Campus-Based need dollars and 
Stafford Loan certification dollars are estimates. Errors in these two programs may not 
translate directly to dollars awarded in error because there is often some need left unmet 
in the Campus-Based programs and students are free to request Stafford Loans for 
amounts less than the full certification amount. 

As shown in Exhibit H-l, the percentage of dollars that are in error is lowest for Stafford 
Loan overcertifications, 9.6 percent. In comparison, the percentage of dollars that are in 
error is 9.9 percent for the Pell Grant program and 113 percent for Campus-Based need 
error. 

The percent of recipients with error is also lowest for the Stafford Loan program. Only 
183 percent of Stafford Loan certification recipients have an overcertification error. 
This compares to 51.0 percent of the Campus-Based recipients that have need error and 
28.2 percent of the Pell recipients that have an error. 



8 



EXHIBIT IM 
Page lof2 



ABSOLUTE ERROR IN THE TITLE IV I^ROGRAMS 
AWARJD YEAR 1988-89 



TYPE OF ERROR 



PERCENT OF TOTAL D OLIJVRS IN ERROR 
OVERALL STUDENT 



_ TOTAL 
DOLLARS' 
INSTITUTIONAL Cm billions^ 



Pell Grant 



9.9 



7.5 



2.6 



4.9 



Campus-Based Need 



11.3 



5.7 



6.2 



5.8 



Stafford Loan 
Overcertifications 



9.6 



4.2 



6.0 



9.0 



• Estimated total value of Pell Grant awards, Campus Bascd need, and Stafford Loan certifications for the 1988-89 award year. 
Pell award estimates were provided by Department of Education. Other estimates arc from the 1988-89 Integrated Quality 
Control Measurement Project conducted for the Department of Education by Price Waterhousc in association with Pclavin 
Associates, Inc. and The Gallup Organization. 



ABSOLUTE ERROR IN THE TITLE IV PROGRAMS 



AWARD YEAR 1988-89 



EXHIBIT II I 
Page 2 of 2 



TYPE OF ERROR 



PERCENT OF RECIPIENTS WITH ERROR 
OVERALL STUDENT 



_ TOTAL 
AWARDS' 

(NSTITUTIONAL rin millions^ 



Pell Grant 



28.2 



24.8 



4.6 



3.3 



Campus-Based Need 



51.0 



37.6 



20.9 



2.2 



Stafford Loan 
Overcertifications 



18.3 



11.5 



9.6 



3.2 



' Estimated number of awards made during the 1988-89 award year. Estimates of the number of awards made were provided by 
Department of Education. Other estimates are from the Integrated Quality Control Measurement Project conducted for the 
Department of Education by Price Watcrhousc in association with Pelavin Associates, Inc. and The Gallup Organization. 
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Exhibit n-1 also reveals differences between student and institutional error rates for the 
three programs as follows: 



• Of the three programs, the Pell Grant program is the only program in 
which errors attributable to institutions accounts for a smaller share of 
dollar error (2.6 percent of Pell dollars) than does error attributable to 
students (7.5 percent of Pell dollars). 

• The amount of student Campus-Based need error, 5.7 percent of need 
dollars, is almc. . equal to the amount of institutional Campus-Based need 
error, 6.2 percent of need dollars. 

• The overcertification rate attributable to institutional error in the Stafford 
Loan program is 6.0 percent of the dollars certified, while the 
overcertification rate attributable to student error is only 4.2 percent of 
total certification dollars. 

• For all three programs, the percent of recipients with error is higher than 
the percent of dollars in error. 

• The percent of recipients with error, among the three programs, is highest 
for Campus-Based need error, 51 percent. More Campus-Based recipients 
had need error attributable to students, 37.6 percent, than institutions, 20.9 
percent. 

• The error rate for institutional error in the Pell Grant program is 4.6 
percent of recipients. This rate is much lower than the student error rate 
of 24.8 percent of recipients. 



Although these error rates are relatively low, and have decreased from the error rates 
estimated during previous studies, the IQCMP results still show that there is a significant 
amount of error in the Title IV delivery system. The nature of the student aid delivery 
system makes the system inherently error prone, but some of this remaining error may 
be reduced through further improvements in the procedures and methods used in 
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awarding student financial assistance. The improvement in error rates will rely, in part, 
on the types of restructuring that are completed A summary of our recommended 
actions to reduce error is presented in Section Vn of this report. 
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III. SOURCES OF ERROR IN THE TITLE IV HNANCIAL AID PROGRAMS 
A. Overview 

Marginal error measures the effect of enors caused by individual data items used in 
award calculation and disbursement. This chapter examines marginal error for both 
students and institutions. Student marginal error is calculated by comparing the award 
that would be computed from all reported data to the award that would be computed 
after correctmg a single appUcation item. Institutional marginal error is calculated in a 
similar manner. Although each marginal error represents a component of overall error, 
marginal error will not sum to overall institutional and smdent error because: 

• some errors may cancel other errors, 

• some errors tend to occur in combination with other errors, and 

• some errors have a carryover effect on other errors (i.e., by being in error 
they will, by definition, cause other items to be in error). 

B. Student Marginal Error 

All student marginal errors were calculated using a $50 tolerance, and measured for 
their effect on Pell absolute award, Campus-Based need, and Stafford Loan 
overcertification. Exhibit III-l presents the variables with the highest marginal absolute 
errors (in dollars) across the three programs. Examination of the significant student 
marginal errors led to the following findings: 
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Most Significant Sources of Student Marginal Error 

Across Programs 
Award Year 1988-1989 



100 



I 

u 

j9 

^ G 

ct o 
.S = 
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Campus-Based 
Stafford Loan 




Parents' Student's 
Number 1987 
In College Adjusted 
Gross 
Income 



Parents' Student's Parents' 
Household Household Home 
Size Size Value 



Source Of Error 



Parents' 
1987 

Adjusted 
Gross 

Income 



Student's 
1987 
Other 

Untaxed 

Income 
And 

Benefits 



Parents' 
Real Estate/ 
Investment 
Value 
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(I) As measured by absolute dollar error in Pell Award, Campus-Based Need and Stafford Loan Ovcrcenificailon 

Esiimaics arc from the 1988-89 Iniegraicd Quality Control Measurement Project conducted for the Deparimeni of Education by Price Waierhousc in association with 
Pclavin Assocliilcs and The Oallun OrBanizaiion. 
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• Student's adjusted gross income for the Stafford Loan program accounts for 
the greatest error among those variables reported by all students. This 
variable is also a large source of error in the other two programs. 

• Parents' adjusted gross income is a large source of error in ail three 
programs, but this variable does not account for as much error as does 
student's adjusted gross income. One reason for this difference is that 
parent values are reported for dependent students only. 

• Parents' number in college and parents' household size are associated with 
the highest levels of student error in both the Pell and Campus-Based 
programs and are a large source of error for the Stafford Loan program. 

• Although smdent's household size is also a significant source of error in the 
three programs, the amount of dollars in error attributable to this variable 
is less than the amount of dollars in error attributable to parents' 
household size. 

• Both parents' home value and parents' value of other real estate and 
investments contribute a large amount to dollars in error in all three 
programs. 

• Student's other untaxed income and benefits fpr 1987 are also significant 
contributors to error. 



C Institutional Marginal Error 

All institutional marginal errors were calculated using a $50 tolerance and were 
measured for their effert on Pell absolute award, Campus-Based need, and Stafford Loan 
overcertification. Exhibit in-2 presents the variables with the highest marginal absolute 
errors (in dollars) across the three programs. An examination of the significant 
instimtional marginal errors leads to the following findings: 
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Most Significant Sources of Institutional Marginal Error^^^ 

Across Programs 
Award Year 1988-1989 




□ Pell 

HI Campus^Based 
I Stafford Loan 



Statement Half-Time Cost of Factoring 

of EnrollnKnt Attendance Other 
Educational Aid 
Purpose 



Selective Bachelor's 
Service Degree 
Compliance 



Financial Default/ Independent 

Aid Repayment Under 
Transcript Unusual 

Circumstances 



Source Of Error 
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O ( I ) As measured by absolute dollar error in Pell Award, Campus Bascd Need and Stafford Loan Overcenincaiion 
E rJc Estimates arc from the 1988-89 Integrated Quality Conuol Measurement Project conducted for the Department of Education by Price Waterhousc in association with 
i f Pclavin A ssociates and The Gallup Organization. 
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• Errors in factoring other aid (Le^ using the correct amount of aid 
prefviousiy awarded to the student when packaging a Campus-Based or 
Stafford Loan) represent the largest total dollars in error for the Campus- 
Based and Sts^ord Loan programs. This error is not applicable to the Pell 
Grant program 

• Errors due to students not being enrolled at least half-tune in a 
postsecondaiy institution are significant in the Stafford Loan program. 
This type of error was not applicable in the Campus-Based programs and 
was not observed for Pell Grant recipients. 

• Using incorrect costs of attendance caused a large amount of dollar error 
in all three programs. 

• Most of the other significant sources of institutional marginal error 
involved not having, or keeping, certain required documents. These 
documents included the statement of educational purpose, statement of 
registration for Selective Service, financial aid transcript, and 
documentation of independent status under unusual circumstances. 
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IV. ERROR PROHLE ANALYSIS 

Enor profile analysis uses multivariate models to idemify the characteristics of students 
and institutions that are associated with error. These models are not intended to be 
used by ED to target students and institutions with error. Rather, multivariate models 
help to understand relationships between error and the characteristics of students and 
institutions. This analysis focused on predicting whether Title IV recipients had any of 
the following three types of error: 

• Absolute Pell Program Error, 

• Absolute Campus-Based Need Error, and 

• Stafford Loan Overcertification Error. 

Separate models were developed for student and institutional errors. Both models were 
formulated using the characteristics of both students and institutions that were believed 
to be related to error. For example, the variables examined while formulating the 
student model included: 

• ITic information reported by the student on the financial aid application, 

• The type of aid received by the student, 

• The characteristics of the institution attended by the student (institution 
type and control), 

• The procedures used by the institution to verify student data, and 

• Questions fr jm the student interview about the student's perception of the 
availability of the information needed to complete the application, the 
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complexity of the application, and the amount of help received while filling 
out the application fonn. 

The variables considered during the formulation of the institutional error model included 
characteristics of the institution (e.g., type, control, region, etc.) and information on 
instimtional procedures for processing and checking the award (level of automation, 
procedures for verifying that the correct information was in the student's financial aid 
file, staffing m the financial aid office, etc.). 

A. Error Prediction Model for Student Error 



Exhibit IV-1 presents the variables that were found to predict independent student error 
and the relative contribution of each level of the variables. Exhibit IV-2 presents the 
same information for dependent smdents. Highlights of these two exhibits include. 

• Income has a strong relationship with student error. Independent students 
with income over $15,000 are predicted to have smdent error 18.7 percent 
more often than independent smdents with income under $7,500, and 
dependent smdents whose parents have income over $25,000 are predicted 
to have smdent error 30.8 percent more often than dependent smdents 
whose parents have less than $15,000 in income. 

• Higher error rates were found for smdents or parents who filed a tax 
return, used estimated tax data rather than acmal tax data, had untaxed 
income, or had significant real estate or investment assets. 

• Smdents who received Pell and/or Campus-Based aid were more error 
prone than those receiving only Stafford loans. This result probably occurs 
because Stafford Loan overcertification error occurs less frequently than 
Pell trror and Campus-Based need error. 
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EXHIBIT IV-1 



Error Prediction Equation for Independent Student Characteristics 
Indodes Relative Coatribution to Error Proiuibility (in percent) 



Intercept (-15^) 
Total Student Income 



Tax Form Filed 



$0 - $7^ (0.0) 
$7^ - $15,000 (8i) 
Over $15,000 (18.7) 



Student filed c tax fonn (7.9) 

• Student did not file a tax form (0.0) 

Used Estimated Income Tax Data 

• Used estimated tax data ^en filling in application (10.2) 

• Did not use estimated tax data when filling in application (0.0) 

Student's Untaxed Income 

• Student has untaxed income (3.7) 

• Student does not have untaxed income (0.0) 

Indicator for Pell Award 

• Student received Pell Grant (15.7) 

• Student did not receive Pell Grant (0.0) 

Indicator for Campus-Based Award 

• Student received Campus-Based Award (8.9) 

• Student did not receive a Campus-Based Award (0.0) 

Indicator for Sta£ford Loan 

• Student was certified for Sta£ford Loan (0.2) 

• Student was not certified for a Stafford Loan (0.0) 
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EXHIBIT IV.2 



Error Prediction Equation for Dependent Student Cliaracteristics 
Indttdcs Relative Cootribntian to Error Probability (io percent) 



Intercept (-122) 
Total Parent Income 



Tax Form Filed 



$0- $13,000 (0.0) 
$15,000 -$25,000 (28.9) 
Over $25,000 (30.8) 



Parent filed a tax form (9.9) 
Parent did not file a tax form (0.0) 



Used Estimated Income Tax Data 

• Student or parent used estimated tax data v^en filling in application 
(116) 

• Neitlier student nor parent used estimated tax data ^en filling in 
applicatioo (0.0) 

Net Value of Parent's Real Estate and Other Investmsnts * 

• $0 . $500 (0.0) 

• Over $500 (05) 

Indicator for Pell Award 

• Student received Pell Grant (15.7) 

• Student did not receive Pell Grant (0.0) 

Indicator for Campus-Based Award 

• Student received Campus-Based Award (10.6) 

• Student did not receive a Campus-Based Award (0.0) 

Indicator for Stafford Loan 

• Student was certified for Stafford Loan (5.6) 

• Student was not certified for a Stafford Loan (0.0) 

Includes the vtlue (after debit) of iavettment real eittte, cash, tavinii and checking accounts, and other 
investments. Does not include investments in homes, businesses, or farms. 



« 



i 



ERIC 



21 

3S 



fOCMP Executive Summanr 



B* Error Predictioii Model for Instit«itional Errors 

The error prediction model for institutional errors is shown in Exhibit IV-3. This cxiubit 
shows that institutional error is in large part related to the financial aid received by the 
student. For example, students receiving Campus-Based aid are predicted to be nearly 
19 percent more likely to have an institutional error than those who do not receive 
Campus-Based aid. This is to be expected because institutional error for Campus-Based 
need was much higher than either Stafford Loan overcertification error or Pell award 
error. The relative order of the effects of awards received in the model (Campus-Based 
higher than Stafford Loan, and Stafford Loan higher than Pell) is identical to the order 
of institutional error rates among the three programs. 

Exhibit IV-3 also shows the following relationships with mstimtional error: 

• Students attending proprietary institutions or attending institutions that do 
not recheck institutional records for the required documentation are more 
likely to have instimtional error. 

• Students attending four-year Baccalaureate or Graduate degree granting 
programs were only slightly less likely to have an institutional error than 
those attending other institutions. 
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EXHmiTIV-3 



Error Prediction Model for Institutional Characteristics 
Indndca Relative Cootribntloo to Error Probability (in percent) 



Intercept (-23) 

Institution Control 

• Public or Private (0.0) 

• Proprietary (8.0) 

Institution Type 

• 0-4 Year Program (2.9) 

• 4 Year Baccalaureate and Graduate Programs (0.0) 

Method used to re-check files for documentation 

• Automated system (0.0) 

• All manual systems (2.7) 

• Do not re-check (8.6) 

Indicator for Pell Award 

• Student received Pell Grant (32) 

• Student did not receive Pell Grant (0.0) 

Indicator for Campus-Based Award 

• Student received Campus-Based Award (18.7) 

• Student did not receive a Campus-Based Award (0.0) 

Indicator for Stafford Loan 

• Student was certified for Stafford Loan (7.2) 

• Student was not certified for a Stafford Loan (0.0) 
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V. EFFECT OF VERIFICATION 

A. Integrated Verification Practices 

Department of Education regulations for Integrated Verification require institutions to 
verify certain applicant-reported data for a sample of Title IV applicants. In addition, 
many institutions supplement their Integrated Verification activities with an Institutional 
Verification program. 

TTtiHftr In tegrated Verification, the central processor, Multiple Data Entry processors, 
and certain Need Analysis Servicers use edits developed by ED to selert Title IV 
applicants for verification. If more than 30 percent of an institution's Title IV applicants 
are selected for Integrated Verification, the institution may choose to verify only a 
subsample of those selected. (The subsample must include at least 30 percent of the 
institution's Title IV applicants.) We found that over 84 percent of all institutions verify 
all ED-selected applicants. Furthermore, because those institutions that do not verify all 
ED-seleaed applicants often do not have many more than 30 percent of their applicants 
selected, only 12 percent of the Pell recipients selected for Integrated Verification were 
not verified because of the 30 percent rule. 

When an applicant is selected for Integrated Verification, the institution must collect 
documentation to verify the following items on the financial aid application: 
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• Adjusted gross income 

• U.S. income taxes paid 

• Untaxed income and benefits 

• Household size 

• Number in college 

The Department's Verification Guide provides guidelines for acceptable documentation. 
The allowable documentation varies depending on the smdent's situation. IQCMP 
findings on documentation show that: 

• Income-related items are most commonly documented by a copy of a tax 
return supplied by the apphcant. 

• Household size and munber in college are most commonly documented by 
a signed statement. 

• Documentation was often not found or not collected for income items that 
were reported as zero (e.g., no reported untaxed income) and for minimum 
values for household size and number in college (e.g., when an 
independent single student reports a household size and number in college 
of one). 

B. Institutional Verification Practices 

Nearly three-quarters of post-secondary institutions have an Institutional Verification 
program under which thtj verify more than the minimum required number of smdents 
and/or more than the minimum required data items on the financial aid application. 
Other findings from this analysis include: 
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• Nearly 43 percent of the institutions that verify additional students verify 
all i^licants. 

• One-third of the institutions said they verify applicants when conflicting 
data is submitted, and nearly 12 percent verify applicant data in error- 
prone circumstances. 

• Most institutions (nearly 80 percent) reported they followed the procedures 
required under Integrated Verification when verifying student data. 

C Effect of Verification on Student Error 

During visits to sampled institutions, our data collectors recorded the information 
initially submitted to the institution by each smdent and the information ultimately used 
by the institution to determine the student's financial aid award. Differences between 
these two sets of data are primarily due to verification, although in some instances 
differences are due to corrections initiated voluntarily by students. We used this data to 
compute the effect of Integrated and Institutional Verification on student error. 

Exhibit V-1 summarizes the enor attributable to two sources: (1) incorrea student data 
initially submitted to institutions, and (2) incorrect student data ultimately used by the 
institution to determine financial aid awards. The difference between these two error 
measures is primarily due to the effects of verification. Exhibit V-1 provides error 
measures for three groups of stodents: (1) students selected by the processor for 
verification, (2) students selected by the institution for verification, and (3) students who 
were not verified. Due to data constraints, Exhibit V-1 is based on only those students 
who received a Pell award. The exhibit docs, however, evaluate the effects of 
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Exhibit V 

rm^T VFttTPir ATION ON STUDENT ERROR IN 

THF TTTT F TV PROfiRAMS 
Award Year 1988-89 

1 

PELL RECIPIENTS ONLY 




PROCESSOR SELECTED 
INSTntlTION SELECTED 
NOT VERIFIED 



551 


33.4 


26.7 


-6.7 


12.7 


9.4 


-3.3 


260 


30.9 


20.7 


-10.2 


10.7 


5.9 


-4.8 


858 


25.8 


24.0 


-1.8 


7.8 


6.5 


-1.3 



CAMPUS BASED ABSOLUTE 
NEED ERROR 



PROCESSOR SELECTED 
INSTirimON SELECTED 
NOT VERIFIED 



STAFFORD LOAN 
OVERCERTHICATION 

PROCESSOR SELECTED 

INSTITimciNS^^ 

notverfiedI^I:--'^' 



ARITHMETIC AViRAGE OF 

ABOVE.:-.v..-^:^:r|^S-^:--' 

PROCESSOR SELECTED 
INSTrrUTION SELECTED 
NOT VERIFIED 



274 
144 
492 



359 
155 
549 



N/A 
N/A 
N/A 



47.9 
47.8 
39.7 



33.9 
39.8 
36.1 



14.0 
-8.0 
-3.6 



14.8 
14.4 
10.5 



10.3 
12.9 
9.3 



-4.5 
-1.5 
-1.2 



32.0 
31.0 
25.3 



23.6 
24.5 
23.1 



-8.4 
-6.5 
-2.2 



1. Recipients who did not receive a Pell are excluded. Total sample 1,669 recipients. 

2. EiTor in applicant's initial submission to the institution. 

3. Error in final applicant data used by the institution to calculate award. 
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5.0 
5.7 
5.5 



2.7 
3.9 
4.6 



4.0 
3.6 
3.8 



2.6 
4.1 
3.4 



7.2 
6.7 
5.7 



4.9 
4.6 
4.8 



-2.3 
-1.8 
-0.9 



•1.4 
0.5 
-0.4 



-2.3 
-2.1 
-0.9 



verification on Campus-Based and Stafford Loan awards to students who also received a 
Pell award. Significant findings from this table include the following: 



• The error in Pell awards for smdents selected by the processor for 
verification (and verified by the institution) was reduced firom an initial 
level of 33.4 percent of students to a final level of 26.7 percent, a reduction 
of 6.7 percent In terms of award amount, the percent of dollars in error 
was reduced from 12.7 percent to 9.4 percent. 

• Error in Pell awards selected for verification hv the institution was reduced 
from 30.9 percent of the students to 20.7 percent of the students, and from 
10.7 percent of the award dollars to 5.9 percent of the awards dollars. 



The above analysis considers errors in all student data items, whether or not they are 
checked as a part of Integrated Verification. Errors in only those smdent items that are 
required to be verified under Integrated Verification are lower in magnitude, but 
generally similar, to those in Exhibit V-1. 



D. Targeting of Verification 



As shown previously in Exhibit V-1, the initial error among students selected by 
processors for Integrated Verification did not differ substantially from that of students 
not selected for verification. Regression models were used to predict whether or not an 
enor occurred based on the applicant-reported data. Results indicate that the following 
two variables were significant predictors of error: 



• Dependent students are nearly three times as likely as independent 

students to have an error in the data initially received by the institution. 
However, the processing edits select dependent students for Integrated 
Verification only slightly more frequently than independent smdents. 
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• Applicants with high AGI ($35,000 parent income for dependents and 
$15,000 income for independents) are more than 6 times as likely to have 
an error in the information initially submitted to the institution than those 
with low AGI. The processing edits select dependent students with high 
AGI less frequently than those with low AGI. The edits select nearly one- 
quarter of the independent students with AGI less than $10,000 - those 
with the least amount of error. 



These findings suggest that ED processing edits could be improved by targeting 
dependent students more frequently than independent students and targeting Iiigh AGI 
smdents and parents more frequently than low AGI students and parents. Because our 
database only includes Title IV recipients, while the selection system is applied to all 
applicants, it is not possible for us to fully evaluate the effectiveness of the selection 
system. 
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VI. SIMPUnCATION OF THE FINANCIAL AID FORMULA 

Simplification of the Title IV Student Financial Aid Delivery Systems has been under 
discussion for several years. Members of the financial aid conmiunity, including ED 
officials, financial aid officers, and members of Congress, all agree that simplification is a 
worthwhile goal. However, an agreement has yet to be reached on the specifics of 
simplification. 

This chapter presents a summary of our analysis of a proposal by the National 
Association of Student Financial Aid Advisors' (NASFAA's) Need Analysis Standards 
Committee (NASC) to simplify the Title IV financial aid formulas. We have modeled 
the effect of the proposed changes to the Pell Grant Program because the effect of the 
changes could be easily traced to changes in Pell award amounts. 

A* Revising the Definition of Dependency Status 

In order to st) eamiine the determination of dependency status, NASC developed an 
alternative definition of dependency status under which a recipient was considered 
independent if they met at least one of the following criteria: 

• Were bom before January 1, 1965 (for the 1988-89 award year) 

• Were an orphan or ward of the court 

• Had legal dependents other than a spouse 

• Were a graduate or professional student 
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• Were a veteran of the U.S. Armed Forces 

Any recipient not meeting at least one of these criteria is considered dependent under 
the alternative definition. 

This alternative definition of dependency status has several desirable features. It is 
simpler than the current definition, requires fewer questions, and eliminates complicated 
branching. In addition, the alternative definition uses easily verified data items. 

To estimate the effect of the alternative definition of dependency status, we determined 
the dependency status for each recipient using the alternative definition, and compared 
this dependency status to the dependency status determined using the current formula. 
A summary of the results of this comparison is shown in Exhibit VI- 1. Significant 
fmdings from this analysis include: 

• Nearly 96 percent of our sampled recipients had the same dependency 
status under the current and alternative definitions. 

• Only 03 percent of recipients were dependent under the current formula 
and independent under the alternative formula. These students were all 
graduate students who were claimed as dependents by their parents for tax 
purposes. 

• 3.9 percent of recipients were independent under the current formula and 
dependent under the alternative formula. 

We analyzed the 4.2 percent of students who changed dependency status between the 
current and alternative definitions. One important group of students who changed 
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Exhibit VI-1 

Effects on Dependency Status of Modifying 



the Dependency Status Definition 

Median Median 

Current Alternate Percent of Student Student 

JEoimula Formula Students Agg losQlDS. 

Dep. Dep. 49.6 20 $1,836 

Dep. Ind. 0.3 23 $4,124 

Ind. Dep. 3.9 22 $6,001 

JmL JmL M2 22 iSJ52fi 

All AU 100.0 22 $3,889 



dependency status was the group of married students. Modifying the alternative 
definition to include all married students as independents would eliminate most of the 
dependency change^ 'or married students and would reduce the overall number of 
dependency status changes from 4.2 percent to 3.2 percent. 

B. Other Changes in Pell Grant Formulas 

The alternative Pell Grant formula includes seven other changes: 

• Eliminating medical and dental expenses from the formula. 
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• Fliwimating elementary and secondary tuition from the formula. 

• Fiiminating the simplified formula. 

• Eliminating work income from the formula. 

• Making AFDC recipients automatically eligible for a full award. 

• Eliminating assets from the formula for lower income families (i.e., 
families meeting both of the following conditions: (1) they filed a 1987 tax 
return on Form 1040A or Form 1040E2^ and (2) they had less than 
$25,000 in total income). 

• Using a calculated value of U.S. taxes paid (using tax tables and reported 
filing stams, number of exemptions, and adjusted gross income) radier than 
the reported figure. 

To evaluate the effect of the alternative formula, we calculated Pell awards using the 
alternative formula and compared them to Pell awards calculated using the current 
formula'. When the alternative definition of dependency status and the above seven 
modifications to the award formula were implemented, 79.5 percent of the Pell Grant 
recipients in our data base had a change in their Pell Grant of $50 or less, and over 95 
percent had a change of $250 or less. Exhibit VI-2 presents a table of changes by award 
amount category. 



Other findings from this analysis include: 

• More Pell Grant recipients had an increase in their award under the 

alternative formula than had a decrease in their award (13.3 percent had 



' We could not calculate alteraative awards for most of the recipients who switched from independent to 
dependent because we did not have the parent information necessary to calculate Pell award amounts. Also, 
recipients categorized as dislocated workers or displaced homemakers were excluded from our analysis because 
their award would be determined through professional judgment under the alternative formula. 
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Ejthibit VI-2 



Distribution of Pell Awards 
Under Current and Alternative SAI Formulas 
(Number of Sampled Recipients) 



AWARD 
UNDER 
ALTERNATE 
FORMULA 




AWARn TINDER CURRENT FORMULA 




$500 $501 - 
OR LESS $1,000 


$1,001- 

S 1.500 


1,501- 
$2,000 


OVER 
$2,000 




$500 or Less 


155 


13 




1 


1 


172 


$501 - $1,000 


15 


245 


10 


1 


1 


272 


$1,001 - $1,500 


2 


26 


332 


5 


1 


366 


$1,501 - $2,000 


0 


1 


13 


273 


10 


297 


Over $2,000 


0 


0 


2 


15 


380 


397 


AU 


172 


285 


359 


295 


393 


1,504 
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an increase in their award of at least $51 while 7.4 percent had a decrease 
in their PeU award of at least $51). 

Nearly 97 percent of recipients with awards over $2,000 under the current 
formula also had awards over $2,000 under the alternative formula. Thus, 
very few of the neediest recipients had a change in their award as a resuh 
of the alternative formula. 

Using a calculated rather than reported amount for U.S. taxes paid 
affected a higher percentage of Pell Grant recipients than any other single 
change modeled, 7 percent. 

The modifications involving AFDC recipients, low income recipients, and 
work income, caused less than 2 percent of recipients to have changes in 
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their Pell award of more than $50. In addition, for each change, a higher 
number of recipients had their awards increase than decrease. 

• The modifications for medical/dental expenses and elementaiy and 
secondary tuition had similar effects on PeU Awards. These changes 
decreased awards for 2.6 percent (elementary and secondary tuition) and 
32 percent (medical/dental expenses) of recipients. 
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VII. CORRECTIVE ACTIONS 
A. Introduction 

This section presents corrective actions for ED to consider in its efforts to reduce error 
in the Title IV student financial assistance programs. These recommendations are based 
on both the findings of the IQCMP and on recognized management practices used by 
other organizations within and outside of the Federal government. 

1. Framework for Corrective Actions 

Corrective actions with regard to the Title IV delivery system should be aimed towards 
increasing the quality of services provided by ED. There are three distinct groups served 
by ED: students who are potential recipients of Titie IV aid (and their families), 
educational institutions that participate in the program, and taxpayers who pay for the 
program. Each of these three groups has specific goals that must be met by ED. ED 
can increase service to students and instimtions by increasing the amount of services 
available, decreasing turn-around time, providing clearer and more in-depth information, 
and reducing the burden involved in the financial aid system. At the same time, ED 
needs to ensure that taxpayer money is being spent correctly (i.e., with a minimum of 
errors), equitably, and in a cost-effective manner. 

Because of the complex nature of the student aid delivery system, errors in awarding 
student financial assistance will never be entirely eliminated. Some reduction in error 
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rates could be achieved by implementing minor modifications to the delivery system (e.g.» 
improving instructions, redesigning forms, etc), but many of these "quick fixes" have been 
tried in the past with limited success. ED may need to consider more sweeping changes 
in the financial aid delivery system, such as simplification of the financial aid formula, in 
order to achieve major reductions in error rates. 

2. Understanding the Causes of Error 

Both students and institutions have motives to maximize the student's Federal aid 
awards. Smdents have the obvious motivation of reducing the financial burden of the 
costs of postsecondary education. Instimtions also benefit fi-om Federal student aid, 
either through reduced demands on institutionally funded financial aid or through 
increased enroUment by financially needy students. These pressures to increase Federal 
aid awards may lead to intentional errors. 

Not all of the error in the Title IV programs can be attributed to intentional errors. 
Most students and institutions are honest and intend to supply complete and accurate 
information, but mistakes are still made. Unintentional mistakes can affect awards 
either in a random fashion (e.g., an unintentional error in home value could increase or 
decrease the award) or in one direction (e.g., forgetting to report untied mcome will 
increase the award). Overall, unintentional mistakes probably tend to increase the 
student award. 
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Although ED has uncovered several cases of fraud in recent years, it is nearly impossible 
for a study such as IQCMP to differentiate between the effects of intentional and 
unintentional errors. We uncovered only one case that was clearly an example of fraud, 
that of a student's submission of a falsified application for financial aid. For this reason, 
the corrective actions presented in this report are designed to reduce both intentional 
and unintentional error. 

B. Proposed Corrective Actions 

Five different strategies for reducing error in the student assistance programs are 
discussed below. These five strategies are intended to cover both intentional and 
unintentional errors committed by smdents and instimtions. The five strategies include: 

• Improv; communications with students and institutions. 

• Remove opportunities for error. 

• Provide disincentives for noncompliance. 

• Identify and correct errors after occurrence. 

• Enlist educational institutions in efforts to reduce error. 

These corrective action strategies, and proposed corrective actions, are discussed below. 

1* Improve Communications With Students and Institutions 

This corrective action strategy includes clarifying instructions, providing more 
information on policies and procedures, and correcting incorrect perceptions. This 

38 



inCMP Exccutivft Smnmarv 



corrective action strategy is relatively easy and low-cost to implement However, because 
of their nature, it is likely that the changes proposed under this strategy will have only 
small effects on error in awarding aid. 



Three specific corrective actions that fall within the overall strategy of improving 
communications with students and instimtions include: 



• aarifv Definitio ns of Student Reported Items -- During IQCMP, several 
definitions on the financial aid application were found to be ambiguous. In 
particular toe definitions for displaced homemaker and household size 
should be clarified. 

• Qarify Procedures For Re porting Changes in Student Situation - The 
necessity for reporting changes in a student's financial situation is not 
spelled out clearly on the financial aid application. Student's should be 
clearly told under what circumstances they must report changes to the 
financial aid administrator. 

• Clarify Repilf ^tinns Regarding the Student Assistance Programs - During 
IQCMP, two areas were uncovered in which ED has given the institutions 
either unclear or erroneous instructions about the policies involving the 
Title IV programs. These two areas involved Stafford Loan 
undercertifications and the worksheets used by the instimtions to calculate 
family contributions. These two areas should be straightened out and 
specific instructions given to the institutions. 



2. Remove Opportunities for Error 



This corrective action strategy is aimed at reducing errors before they have an 
opportunity to occur. The corrective actions suggested as part of this strategy require 
more substantive changes in the delivery system than do the other strategies. Because 
this strategy should eliminate opportunities for error, the error rates should decrease. 
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Hiree specific corrective actions that are designed to reduce opportunities for error are 
as follows: 



• Simplificatinn nf the Financ ial Aid Formula - This corrective action, which 
has been discussed for several years in various forms, will involve a major 
investment in an overhaul of the financial aid delivery system. Chapter VI 
of this report discusses a simplified student aid formula proposed by 
NASFAA. 

• Revise Reporting Procedure s for Pell Cost nf Attendance - During the 
processing of IQCMP data, several cases were found to be in error because 
the institution failed to apply the regulatory limits to the components of 
cost of attendance in the Pell program. Requiring schools to report each 
item within the cost of attendance separately or eliminating the ceilings 
placed on the components of Pell cost of attendance would remove the 
opportunity to make this type of error. 

• Do Not Disburse Awards Based on Estimated Tax Data - Estimated tax 
data are error prone and difficult to verify. Althougii a student may need 
to file a financial aid application before a tax return is complete, the 
student usually will not receive disbursed funds until after a tax form is 
filed. We reconunend that funds not be disbursed until after a student 
updates estimated tax data. 

3. Provide Disincentives for Noncompliance 



This corrective action strategy involves creating the perception that data will be checked 
and that the potential for stiff penalties more than offsets the benefits of error. To be 
My successful, this strategy requires a delivery system that holds both students and 
institutions accountable for error and that uses only data elements that can be verified. 



This strategy is primarily aimed at reducing intentional errors. Applicants who 
deliberately report incorrect information will not be as likely to do so if they perceive 
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that they will be caught and punished. A secondaiy benefit of the strategy is potential 
reduction in unintentional errors because people will be xuore careful in completing the 
application if they know they will be penahzed for an error. 



Specific corrective actions that implement the strategy of providing disincentives for 
noncompliance are as follows: 



• Require Specific Information on Ho usehold Size and Number in College - 
The IRS has found that requiring taxpayers to complete a table 
documenting dependents (e.g., name, social security number, etc.) improved 
compliance with the tax laws. ED should consider using a table on the 
application to solicit information on household size and number in college 
(e.g., name, age, relationship, etc.). This table could also be used to track 
information during the verification process. 

• Require Specific Information on Hom e Value and Debt - In an approach 
similar to that recommended for household size and number in college, ED 
should collect specific information on home value and debt including 
cunent home value, current home debt, ZIP Code of home, original 
purchase price of home, and year of purchase. 

• A pply Penalties More Freou entlv to Causes With Error - ED should begin 
using fines more often for student applications with incorrect data. 
Applying small penalties to cases with intentional error will reduce future 
error by increasing the perceived likelihood of being caught. Although 
students probably do not view the postsecondary institution as an 
enforcement entity, and although enforcement is not a primary mission of 
ED, the perception of a significant ED enforcement presence is necessary 
to promote voluntary compliance by students. 

4. Identify and Correct Errors Alter Occurrence 



This conective action strategy involves verifying data that has already been collected and 
ensuring that ED procedures are being followed by the institutions. The corrective 
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actions in this strategy are relatively higher in cost since they involve personnel to check 
and correct past actions, instead of eliminating errors before they occur. However, it is 
important that ED maintain an oversight presence fr instimtions and students. 

The Department of Education has akeady implemented conective actions using this 
strategy. Integrated Verification focuses on identifying student errors, and an 
institutional auditing and review policy focuses on uncovering institutional errors. 
However, as discussed in Chapter V on verification, it appears that targeting students for 
verification could be significantly improved. A full review of verification procedures was 
beyond the scope of IQCMP. 

5. Enlist Educational Institutions in ECTorts to Reduce Error 

Many institutions already have a strong commitment to reducing error in the Title IV 
programs. Most institutions (over 84 percent) verify all ED-selected applicants, and 
almost 73 percent of institutions have institutional verification policies (i.e., smdents arc 
verified who were not selected by ED). Nearly 43 percent of the institutions performing 
institutional verification say that they verify all students. This commitment to quality 
control should be encouraged by ED. 

Quality control is implemented in the Title IV delivery system by assessing penalties for 
errors that are made by the institutions. There are few incentives in place to encourage 
institutions to improve the quality of financial aid delivery beyond a level that is needed 
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to avoid penalties. Under the current regulations, all schools are treated equally, 
regardless of their past performance as measured by error rates. Institutions that 
understand and apply quality control principles, and thus achieve lower error rates, may 
be spending too much time fulfilling regulations; some of this time might be better spent 
ia implementing error reduction procedures geared toward tiieir own institution. In 
order for a program such as this to be a success, monitoring the error rates for these 
institutions would be needed to ensure that quality control practices are not slipping. 

One promising program that involves institutions in efforts to reduce error is the 
Institutional Quality Control Pilot Project. This project is still a relatively new project. 
It has been successful in recruiting institutions to participate, but reduction in error rates 
have not been as large as desired. The Pilot project represents a promising quality 
control program focusing on management practices and should be refined and continued. 

Institutions that are not approved for, or that choose not to participate in, the Pilot 
project should also receive ED guidance in using quality control procedures to improve 
financial aid management. ED, through its training and dissemination offices, could 
provide a series of brochures or booklets on quality control procedures that institutions, 
especially tiiose in tiie Pilot project, have successfully implemented. In addition, ED 
should continue to encourage institutions to develop quality control procedures and 
adopt procedures found to reduce error at other institutions. ED could promote 
commitment to quality control through its "Dear Colleague" letter and encourage schools 
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to network with each other through NASFAA and other professional organizations. 
C Conclusions 

The five corrective action strategies discussed above range in scope from minor to whole 
sale changes of the delivery system. Exhibit Vn-1 presents a summary of the extent of 
modifications to the delivery system required to implement each conective action 
strategy . The corrective actions suggested under each of these strategies can be 
implemented in any combination that is deemed appropriate to meet the Department's 
goals. 

Despite the decrease in error rates found during IQCMP, error in the Title IV system is 
still significant. ED should review its goals for quality control, as well as the corrective 
actions discussed above, and develop a plan for reducing error rates consistent with its 
goals for providing quality service. 
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EXHIBIT Vn-1 



EXTE^^^ of modmcations to the title iv delivery system 

REQUIRED BY CORRECTIVE ACHON STRATEGIES 



^tivm Action Strategy 



Improve Communications With 
Students and Institutions 

Remove Opportunities for Error 

Provide Disincentives for 
Noncompliance 

Identify and Correct Errors 
After Occurrence 

Enlist Educational Institutions in 
Efforts to Reduce Error 



Extent of Modincations 



X 



X 



X 
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L INTRODUCTION AND BACKGROUND 

A. The Integrat d Quality Control Measurement Project 

1. Background and Objectives 

The purpose of the Integrated Quality Control Measurement Project (IQCMP) is to 
evaluate the quaUty of the delivery of Title IV student financial assistance to students 
attending postsecondary institutions during the 1988-89 academic year. The Title IV 
programs evaluated in the study include: 

• The Pell Grant Program 

• The Campus-Based Programs (Supplemental Educational Opportunity Grant, 
College Work-Study, and Perkins Loan), and 

• The Stafford Loan Program (formerly the Guaranteed Student Loan Program) 



IQCMP is the most recent in a series of quality control studies conducted by the U.S. 
Department of Education (ED) to determine the quality of the delivery of Title IV 
funds. The study was sponsored by the Statistical Analysis Branch of the Division of 
Quality Assurance in the Debt Collection and Management Assistance Service of the 
Office of Postsecondary Education, U.S. Department of Education. The study was 
conducted by Price Waterhouse in affiliation with The Gallup Organization and Pelavin 
Associates, Inc. 
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This report presents findings on error in the delivery of Title IV student financial aid 
and proposes corrective actions aimed at reducing error in future years. Other project 
reports including the lOCMP Error Definitions and IOC Pilot Analysis present in-depth 
discussions of the error definitions and an analysis of the Institutional Quality Control 
Pilot Project, respectively. 

2. IQCMP Methodology 

The IQCMP relied on a nationwide sample of students to make inferences at the 
national level about the quality of the award process for the various Title IV financial 
aid programs. The sample was designed to be large enough to make statistical 
inferences about the percent of awards in error for each Title IV program with +/- 5 
percent accuracy at a 90 percent confidence level. The sample data cannot be used to 
make reliable estimates of error at any particular postsecondary institution. 

A random sample of 300 public, private, and proprietary instimtions was selected from 
among all institutions in the continental U.S. participating in the Title IV student 
financial aid programs. These institutions, plus the 50 institutions participating in the 
Institutional Quality Control Pilot Project, formed the basis for the study sample. 

Data collection began with visits to the sampled institutions. First, financial aid 
administrators were interviewed to understand each institution's financial aid policies and 
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procedures. Next, a random sample of students participating in the Title IV programs at 
each institution was selected. A total of 3,310 students were selected at the 350 
institutions visited. Finally, information from the school's financial aid records was 
abstracted for each of the sampled students. Inconsistencies in the institution's financial 
aid records were discussed with the financial aid administrator to obtain an explanation 
or to confirm an error. 

After the instimtional data collection, interviews were conducted with sampled students 
and their parents. During these interviews, students and parents were requested to show 
documents verifying the data reported on the smdent's financial aid application. 
Additional information from sources such as the IRS, financial institutions, and property 
tax assessors was also collected to confirm or refute data reported on the student's 
financial aid application. 

Information from all sources was combined into a comprehensive database. During this 
step, cases that were missing a student interview or a dependent student's parent 
interview were dropped from the analysis. Our final analysis database contained 2,653 
students. This database was used to calculate "best" award values (based on the most 
reliable information collected from among the vsrious sources) and to make projections 
of differences between best awards and actual awards received. 
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The results presented in this report are based on abstracts of 2,653 student records, 2,653 
student intemews, 1,594 parent interviews, 2,440 tax forms, 127 financial institution 
records, and 500 property tax assessor records. 

B. Background on the Title IV Financial Aid Deliveiy System 
1. The Title IV Programs 

The Department of Education is responsible for administering Federal financial aid to 
students enrolled in postsecondary educational institutions. The majority of these 
programs are authorized under Title IV of the Higher Education Act (HEa) of 1965 (20 
use 1070a, b, 1071, 1087a, and 2751; and 42 USC 2751) and its amendments. The most 
recent amendments are the Higher Education Amendments of 1986 (P.L 99-498) and 
the Higher Education Technical Ariendments act of 1987 (P.L. 100-50). 

There are five major Federal student financial aid programs that are commonly referred 
to as the Title IV programs. These five programs, examined during IQCMP, are: 

• The Pell Grant Program ~ The Pell Grant (formerly Basic Education Opportunity 
Grant) Program is a Federally-funded formula grant program. A student's Pell 
Grant award is determined based on a legislated formula and is not dependent on 
the amount of money available at the institution. Only undergraduate students 
attending participating institutions at least half time are eligible. 

• The Supplemental Education Opportunit v Grants (SEOG^ Program - SEOG is a 
Federally-funded grant program targeted to students who show exceptional need, 
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with preference given to students who also receive Pell Grants. SEOG grants are 
determined and awarded hy the institution. The amount of an SEOG is 
determined, in part, on the amount of money available at the institution. Only 
undergraduate students attending participating institutions are eligible. 

• The College Work-Study (CWS) Program -- CWS is a program to distribute 
money to eligible students, both undergraduate and graduate, in exchange for 
part-time work. CWS programs are implemented by the institution, and student 
awards are determmed based on the amount of money available at the institution. 
In most cases, the Federal government supplies 80 percent of CWS funds. 

• The Perkins Loan Program - Perkins Loans (formerly National Direct Smdent 
Loans) are low-interest, long-term loans distributed to both undergraduate and 
graduate students. Perkins Loans are made by the institutions, and award 
amounts are determmed based on the amount of money available at the 
institution. Institutions supply $1 for every $9 the Federal government supplies to 
the Perkins Loan funds. 

• The Stafford Loan Program - Stafford Loans (formerly Guaranteed Student 
Loans) are low interest, long-term loans made by lenders and subsidized and 
guaranteed by the Federal government. Both undergraduate and graduate 
students who are enrolled in eligible programs at least half time may receive 
Stafford Loans. 

Together, the Supplemental Education Opportunity Grant, College Work-Study, and 
Perkins Ix)ans programs are grouped as Campus-Based Programs. Although these 
programs are largely financed by the Federal Government, institutions may establish 
their own policies for distributing the limited funding among eligible applicants. 



The Title IV programs distributed over $15.6 billion in financial aid to over 5.5 million 
students during the 1988-89 award year. Exhibit I-l presents a breakdown of dollars and 
awards by program. Over 8,000 postsecondary institutions and more than 10,000 lending 
institutions and guarantee agencies participate in the delivery system. 



1-5 



ERIC 



G7 



TITLE IV AID AVAILABLE TO STUDENTS 

Award Year 1988-89 




Pell Grants 


4,863 


3,302,000 


1,473 


Campus-Based 








Woik Study 


780 


835,000 


934 


SEOG 


442 


633,000 


698 


Perkins Loan 


884 


826,000 


1,070 


Total Campus Based . 


2,106 


2,294,000 


918 



Stafford Loans 



8,431 



3,234,000 2,607 



Total 



15,400 



8,830,000 1,744 



Data from 1991 budget request provided by the Department of Education 
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2, Roles of Participants in Cumnt Financial Aid Deliveiy System 

The Title IV Federal financial aid delivery system has four xr^ajor participant 
groups: 

• The Department of Education (ED), 

• Smdents applying for Title IV financial aid, 
» The Tide IV processors, and 

• Financial Aid Administrators 

We summarize the roles of each participant group below. 

a. The Department of Education 

The Office of Postsecondary Education within the Department of Education has the 
responsibilit)' of administering the Title IV financial aid programs. ED develops and 
analyzes financial aid policies, programs, and regulations, develops policy manuals and 
application forms, and develops budgets and analyzes the impact and cost of initiatives. 

ED is also responsible for contracting with need analysis servicers (organizations that 
calculate student need and provide institutions with other financial aid services) to 
become Multiple Data Entry (MDE) processors. MDEs may be used in place of the 
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Federal Application Processing Center to calculate the student's (and his or her famil/s) 
expected contribution to education, as expressed by either the Student Aid Index (SAI) 
for the Pell Grant Program or the Family Contribution (FC) for the Campus-Based and 
Stafford Loan Programs. 

b. Students 

The student is responsible for completing the application for Federal student aid 
completely and accurately. If the student is financially dependent on his or ner parents, 
he or she must also provide information on the parents' financial status. Hie student 
must provide documentation for selected data items, if requested. 

To be eligible for Title IV student financial aid, a student must be: 

• A U.S. citizen or eligible non-citizen, 

• Registered with Selective Service if required, 

• Enrolled in an eligible institution and program, 

• Making satisfactory academic progress, 

• Not in default on a loan or owe a refund on any grant from a Title IV program, 
and 

• Financially needy, as determined by predefined standards. 
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c Federal Aid Processors 

The Federal Processing System is composed of the Federal Application Processing 
Center in Iowa City, Iowa, and several MDE processors and need analysis servicers. 
Information supplied by the student on either the Federal application or an application 
specific to an MDE is used by the processor (i.e., the Federal Processor, an MDE, or 
some other need analysis servicer certified by the Department of Education) to calculate 
the SAI and FC using the Pell Grant Formula and the Congressional Methodology, 
respectively. If the student requested a Pell Grant, the processor transmits the student's 
original application information to the Federal processor. The Federal processor sends a 
Student Aid Report (SAR), containing the financial information supplied on the 
application, to the student. The student is then responsible for reviewing the information 
on the SAR and supplying updated information when necessary. The Federal processor 
and other certified processors are responsible for selecting students for Integrated 
Verification, the process of verifying certain items from the student's aid application. 

d. Financial Aid Administrators 

The financial aid administrator (FAA) is responsible for overseeing the administration of 
the Title IV student aid programs at the educational institution. The FAA determines 
the student's eligibility for Title IV aid, calculates the student's need, and determines the 
amount of financial aid the student will receive from various sources following ED and 
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institutional guidelines. In addition, the FAA is responsible for verifying the information 
supplied by students selected by the processor for verification and for ensuring that the 
proper amount of money is repaid or refunded to the appropriate program if a student is 
no longer eligible. 

FAA's also have the authority to override or adjust information used in calculating the 
student's aid, including the FC and the cost of attendance. During the 1987-88 award 
year, FAA's were given the power to override the student's dependency status from 
dependent to independent if the reason for the override was documented. In addition, 
for the 1988-89 award year FAA's were given the authority to adjust the SAI for students 
with special circumstances. This authority was withdrawn for the 1989-90 award year. 

3. Flow of Data Through the Title IV Delivery System 

The Title IV delivery system is structured to meet the diverse needs of students 
attending postsecondary institutions. The four basic components of the system, described 
above, can interact in various ways. Exhibit 1-2 presents a simplified view of the data 
flow in the Title IV system. 

To be eligible to receive Title IV financial assistance, a student must demonstrate 
financial need. Financial need is the difference between the program's cost of 
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attendance and the ability of the student (and his or her family) to meet those costs. 

As the first step in showing financial need, the student and his or her family complete an 
application for financial assistance. This application can be either the Federal 
application, or an application used by one of the MD£ processors. On the appUcation, 
the student answers questions about his or her dependency status (i.e., does the student 
rely on parental support), income, assets, and expenses. This appUcation is then sent to 
either the Federal processor, an MDE processor, or a needs analysis servicer certified by 
the Department of Education. Because different institutions contract with different 
processors, a student who is applying to more than one school may fill out several 
different applications, all asking for essentially the same informatin. 

The processor enters the information contained on the student's application into a data 
processing system. The system checks the student data for internal consistency and 
completeness. An application that does not pass this editing process is rejected and 
returned to the student. The student makes any necessary corrections and returns the 
updated application to the processor. This procedure continues until the student data 
passes the editing process. 

A key element in determining financial need is the financial contribution expected from 
the student and family, called the Student Aid Index (SAI) for the Pell Grant program 
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and the family contribution (FC) for the Campus-Based and Stafford Loan programs. 
This contribution is calculated by the processor from the student application using 
formulas set by Congress. The processor also uses an automated system to select 
students for Integrated Verification. Students are selected for verification using criteria 
set by ED. 

The financial aid administrator at the institution uses the student's SAI or FC in 
combination with the institution's information on cost of attendance, other aid awarded 
to the studer , and institutional policies to derive a package of financial aid for the 
student. The FAA may use professional judgment to change one or more of the items 
that is used in calculating ^nancial aid. 

The FAA must also ensure that the student meets the eligibility criteria for financial aid 
and that all of the student's documentation is in order before money is disbursed to the 
student. This step includes checking that necessary Financial Aid Transcripts (FAT), 
statements of Selective Service registration, Statements of Educational Purposes, etc., are 
on file. Additionally, the FAA must verify selected data items for those students selected 
for Integrated Verification. After the educational institution disburses aid to the student, 
it reports the disbursement to ED and is reimbursed for the Federal portion of the 
award. 
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As may be seen from the above discussion of the Title IV delivery system, the student 
data used in calculating the financial aid award is checked at several different points. 
Much of the error that could occur in awarding financial aid is removed during the 
award process. Some of the error is also removed during verification. The error that 
has been examined during IQCMP is the error that remains in the award after it has 
been processed by the delivery system. IQCMP measurement of this error is described 
below. 



C. Error in IQCMP 



1. IQCMP Error Measurement Philosophy 



The objective of IQCMP is to measure the quality of the financial ;iid delivery system 
and to identify steps that can be taken to improve quality. The term "error" as defined 
in IQCMP therefore refers to a deficiency in quality, which is a much broader definition 
of error than that included in the Title IV regulations. 

In general terms, error is defined to be the extent to which the delivery of Title IV aid 
differs from that intended under the Title IV regulations. This definition includes the 
following types of error: 
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• Institutional Liability Error - A limited group of errors defined in the Title IV 
regulations. Institutions who commit liability errors must reimburse the Federal 
government for the amount of the error. 

• Instimtional f ailure to follow the Title IV regulations - A failure on the part of 
the institution to follow the Title IV regulations, when that failure is not a liability 
error. These errors include incorrect calculation of the financial aid award and 
failures to follow regulatory procedures. 

• Instimtional failure to follow its own a ward policies - A failure by the instimtion 
to follow its own policies for Title IV aid, even when that failuie does not violate 
the Title IV regulations. 

• Intentional student reporting errors - An intentional failure by the smdent to 
report the correct data on his or her financial aid application. Under the Title IV 
regulations, students are held accountable for these errors and may be subject to 
fines and/or imprisonment. 

• Unintentional student reporting errors - An unintentional failure by the smdent 
to report the correct data on his or her financial application. Under the Title IV 
regulations, smdents aid awards should be adjusted if unintentional errors are 
come to the attention of the institution during verification of the smdent aid 
application. However, the smdent is not subject to fines or imprisonment,. 

• Errors in estimates and projections - The award calculation relies in part on 
estimates (e.g., estimates of the value of a student's home or real estate 
investments) and projections (e.g., the expected household size or expected 
outside financial assistance for the upcoming academic year). Incorrect estimates 
and projections, if made in good faith, do not represent errors under the Title IV 
regulations. However, IQCMP includes incorrect estimates and projections as 
errors in order to evaluate the extent to which financial aid awards are consistent 
with the intent of the Title IV program. 



The following examples help illustrate how "error" measured during IQCMP differs from 
error under the Title IV regulations. 
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Example J 

In order to distribute limited Campus-Based funds to as many students as 
possible, the University of Anytown has established a policy that Campus-Based 
awards cannot exceed $1,500. Jane Student has a Campus-Based need of $4,000 
and receives a Campus-Based award of $2,000. For the purposes of this study, 
we compute an error in Jane's Campus-Based award of $500 because the $2,000 
award exceeds the institution's policy of a maximum $1,500 award. From a 
regulatory standpoint, no error has occurred because the Title IV regulations do 
not limit Campus-Based awards to $1,500. 



Example 2 

John Smith, a sophomore, applies for financial aid as a dependent student. 
Because John's parents expect his younger sister Jackie to start college in the fall, 
they report that the number in college for his parent's household is two. Raising 
the number of Smith dependents in college to two makes John eligible for a $750 
Stafford Loan. Two weeks before classes start, Jackie decides to defer her 
enrollment. For the purposes of this study, we compute an overaward error of 
$750 for John's Stafford Loan. No error has occurred from a regulatory 
standpoint. 
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Example 3 

Joe Student is attending the College of State Undergraduates as a sophomore. 
Over the previous summer, Joe turned 18 and registered with Selective Service. 
For his sophomore year Joe applied for Title IV aid and received a $1,500 Pell 
Gi-ant and a $1,800 Stafford Loan. Although the amount of aid awarded to Joe 
was correct, he failed to sigr the required Statement of Registration Status to 
document that he had indeed registered with Selective Service. For this study we 
would determine that a $1,500 error in the Pell award was made and an $1,800 
error in the Stafford Loan award because the institution failed to collect the 
signed Statement of Registration Status required by the Title IV regulations. 
However, under the Title IV regulations, the institution is not liable for the en-or 
because the student did in fact register with Selective Service. 



2. Methodology Used to Calculate Errors 

Error is calculated as the difference between the actual award or need and the "best 
value" award or need computed using data collected from students, parents, institutions, 
and external sources. As the first step in this process, the sources of data available for 
each item are ranked from most to least reliable. The best value for a data item is the 
value associated with the source of highest reliability. 

The table on the following page presents an example of the priorities assigned to sources 
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of parent's adjusted gross income (AGI) data and the frequency with which each source 
was used. 



Example of Best Value Selection of Parent's AGI 

Frequency used 

Priority Source Type of Documentation as Best Value 



1 


IRS 


Copy of Tax Return 


60.0% 


2 


Parent Interview 


Certified Tax Return 


1.3% 


3 


Financial Aid Records 


Certified Tax Return 


0.0% 


4 


Parent Interview 


Copy of Tax Return 


21.0% 


5 


Financial Aid Records 


Copy of Tax Return 


3.5% 


i6 


Parent Interview 


Hand Written Tax Return 


0.0% 


7 


Financial Aid Records 


Signed Statement 


0.0% 


8 


Student Aid Report 


Undocumented 


4.1% 


9 


Financial Aid Records 


Undocumented 


1.8% 








100.0% 



The best SAI or FC is calculated using the best values for all student data items (e.g., 
adjusted gross income, household si;:e, etc.). This best SAI or FC is then combined with 
best institutional data items, collected from the institution site visits (e.g., cost of 
attendance, other aid awarded, etc.), to calculate a best award. The best award is 
compared to the actual amount awarded to the student. Any difference of more than 
$50 between the best and actual awards is considered to be an overall error. 
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Institutional and student error are calculated in a similar fashion. Institutional error 
considers only errors committed by the institution. The best institutional award is 
cdculated using a combination of best institutional data items and student data items as 
reported on the financial aid apphcation or the SAR. This best institutional award is 
compared to the actual award to determine institutional error. Similarly, the best 
student award is calculated using a combination of actual institutional data values and 
best student data as collected during IQCMP. Differences between the best student 
award and the actual award are labeled student error. 



Seven measures of error are used to describe the quahty of the Title IV programs: 

• Composite Error -- error in the total amount of financial support provided to the 
student by the five Title IV programs. The amount of financial support is the sum 
of the Pell Grant award, Campus-Based amount distributed, and Stafford Loan 
amount. 

• Pell Pavment Error ~ error in payments from the Pell program, both under and 
over. 

• Campus-Based Need Error -- error in the calculated student need (the difference 
between the cost of attendance and the sum of the expected family contribution 
and other known financial aid). 

• Campus-Based Distributional Error - an estimate of the payment consequences of 
all need errors (taking into account institutional policies that may limit the extent 
to which the award fully meets student need). 



3. 



Definition of Error Measures 
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• Campus-Based Award in Excess of Need - the extent to which the actual 
Campus-Based award received exceeds the best student need. 

• Stafford Loan Overcertifica tion Error - the extent to which the calculated 
Stafford Loan certification amount (the maximum loan amount available to the 
student) exceeds the best certification amount. 

• Stafford Loan Overaward Error - the extent to which the actual Stafford Loan 
received exceeds the best certification amount. 



Exhibit 1-3 iUustrates the calculation of these seven error measures. 

We compute a Pell Grant award error of $100 ($,1000 - $900 = $100) because the 
actual award disbursed exceeds the best award by $100. Similarly, we will compute a 
Campus-Based need error of $500 ($2,000 - $1,500 $500) and a Campus-Based 
disbursement error of $400 ($1,600 - $1,200 = $400) respectively. Campus-Based award 
in excess of need is the extent to which actual Campus-Based award ($1,600) exceeds the 
best Campus-Based need ($1,500). In this case we compute a Campus-Based award in 
excess of need error of $100 ($1,600 - $1,500 = $100). When the actual Campus-Based 
award is less than the best Campus-Based need, award in ex^^ess of need is set to $0. In 
Exhibit 1-3, the actual Stafford Loan certification ($2,000) exceeds the best certification 
($1,000) by $1,000. Therefore, we calculate a Stafford Loan overcertification error of 
$1,000 ($2,000 - $1,000 = $1,000). 
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Exhibit 1-3 

Example to Demonstrate IQCMP Error 



A. Hypothetical Actual and Best Values 

Actual Bgst 

Pell Grant award $1,000 $900 

Campus-Based need $2,000 $1,500 

Campus-Based award $1,600 $1,200 

Stafford Loan certification $2,000 $1,000 

Stafford Loan $1,000 $1,000 

B. Error Computations Dollar Error 

Pell Grant award error $100 

Campus-Based need error $500 

Campus-Based disbursement error $400 

Campus-Based award in excess of need $100 

Stafford Loan overcertification error $1,000 

Stafford Loan overaward error ^ 

Composite Error $500 



Composite error represents the total amount of financial support in error, and is 
the sum of Pell Grant award error, Campus-Based disbursement error, and 
Stafford Loan overaward error. 
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While IQCMP measures overcertification error, it does not measure undercertification 
error. The Stafford Loan certification calculated according to Title IV regulations 
represents the maximum loan amount that the student can receive. However, students 
sometimes elect to take a Stafford Loan that is less this maximum certification amount. 
We found that when this occurs, institutions usually certified the student for the 
requested loan amount rather than the maximum certification amount. As a result, when 
the acmal Stafford Loan certification was less than the maximum certification amount 
computed during IQCMP, we could not determine whether the institution had made an 
error or the student requested a loan that was less than the full certification. 

The computation of Stafford Loan overaward was also affected by the fact that smdents 
sometimes requested a loan that was less than the maximum certification. When the 
best certification computed during IQCMP was less than the actual certification, it was 
necessary to estimate the loan amount that the student would have elected to receive 
under the smaller certification. We assumed that the student would request a loan for 
the full amount of the best certification. When the best certification computed during 
IQCMP was greater than the actual certification, we assumed the student would request 
a lo3ji up to the amount of the actual loan requested. Exhibit 1-3 illustrates a case 
where the actual certification ($2,000) exceeds the best certification ($1,000). Suppose 
we switch the best and actual certifications, so that the best certification exceeds the 
actual by $1,000. In this case, we assume the student would request the same amount 
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under both certifications ($1,000) and calculate no Stafford overaward error ($1,000 - 
$1,000 = $0). 



Finally, IQCMP will compute a Composite error of $500 for the student in the example 
above. Composite error represents the total amuunt of financial support in error for the 
five Title IV Programs. In the example above, Composite error is composed of: 



Pell Grant award error $100 

Campus-Based disbursement error $400 

Stafford Loan overaward error $ 0 

Composite error $500 



Each of thebe seven types of error listed above is measured using two different error 
sources: 



• Student Error - error resulting from inaccurate data reported by the student 
recipient. Best student awards are calculated using best student values and 
reported institutional values. 

• Institutional Error - error resulting from inaccurate processing by the institutions 
or from improperly applying institution packaging guidelines. Best institutional 
awards are calculated using reported student values and best institution values. 

• Overall Error - error resulting from either student or institutional error as 
defined above. 
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It should be noted that student error and institutional error do not necessarily sum to the 
overall error, because errors made by students and institutions do not necessarily have an 
additive effect on the overall award. Some errors cancel each other, some errors occur 
in groups, and some errors may have a multiplier effect (i.e., one item in error may, by 
definition, lead to another item being in error). All errors presented in this report are 
calculated with a $50 tolerance; that is, a best award within plus or minus $50 of the 
actual award is treated as zero error. 

Composite Error, Pell Grant Payment Error, Campus-Based Need Error, and Campus- 
Based Distributional Error are all reported in the following three ways: 

• Overaward COverstatement't Error - the extent to which the actual award (need) 
exceeds the award (need) computed using best values. 

• Underaward (Understatement^ Error - the extent to which the actual award 
(need) is less than the award (need) computed using best values. 

• Absolute Error ~ the sum of overawards and underawards (where both 
overawards and underawards are considered to be positive). 

Only overawards are reported for Campus-Based Award in Excess of Need Error, 
Stafford Loan Certification Error, and Stafford Loan Award Error. 
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D. Report Organization 

The remainder of this report details our findings on error and the corrective actions that 
we propose. This information is contained in the following chapters: 



n. Overall Error in the Title IV Financial Aid Programs 

III. Sources of Error in the Title IV Financial Aid Program 

IV. Error Profile Analysis 

V. Verification Analysis 

VI. Simplification of the Financial Aid Formula 
Vn. Corrective Actions 
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II. ERROR IN THE TITLE IV FINANCIAL AID PROGRAMS 



A. Composite Error in the Title IV Student Financial Assistance Programs 

Composite error is defined as the actual total award disbursed minus the best total 
award computed using th e best values. The actual total award is considered to be the 
sum of the Pell award, Campus-Based amount distributed, and the Stafford Loan 
amount. Together, the Pell, Campus-Based, and Stafford Loan programs distributed 
approximately $15.4 bilUon in smdent financial aid during the 1988-89 academic year. 
Exhibit I-l (Chapter 1) presented a table of Title IV aid available and the number of 
students receiving awards for each program. 

The results of the composite error analysis are shown in Exhibit II-l. Note that enor in 
this context does not necessarily mean that institutions and students behaved as they 
should not have under applicable regulations. However, many documented data 
variances do have a significant dollar impact and are important quality indicators. 
Fmdings of the composite error analysis mclude the following: 

• Overall absolute error for the three programs was $1.7 billion, or 
approximately 10.9 percent of program dollars. About 36.3 percent of 
smdents receiving Title IV aid had errors over $50. The average absolute 
error per smdent v/ith error was $918. 

• Absolute student error was higher than absolute institutional error (6.6 
percent of dollars and 5.1 percent of dollars, respectively). 

• Mean error per recipient with error was higher for institutional error than 
for smd nt error ($1,125 for institutions versus $742 for students). 

• Overaward errors were significantly higher than underawards for both 
student and institutional errors. 
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COMPOSITE TITLE IV AWARD ERROR* 

Award Year 1988-89 



Exhibit II- 1 



Error Measure 



OVERALL 

Absolulc 

Ovcraward 
Uiulcraward 



STUDIiNT 
Absolulc 

Ovcraward 

Uiulciaward 



INSTITD'I'lONAL 
Absolute 
Ovcraward 
Uiulcraward 




Percent of 
Recipients! 
witli Error 



36.6(±2.5) 
29.4(±2.3) 
7.2(±1.5) 



27.6(±2.4) 
2I.()(±2.1) 
6.6(±1.5) 



13.9(±1.5) 
I2.4(±1.4) 
1.5(±().4) 



Percent ^ 
of Dollars 
in Error 



10.9(±!.0) 
9.8(±'l.()) 
1.1 (±0.3) 



6.6(±().8) 
5.7(10.7) 
0.9(±0.2) 



5.1(±0.8) 
4.8(10.8) 
0.3(10.1) 



Total 

i^mount 
($ Millions) 



1,679 
1,509 
169 



1.016 
878 
139 



785 
739 
46 



I. ('oni|M)siic Tiilc IV award error inchulcs errors in atnoiini awarded to sliideiils for llic Pell Gram, Cam|)u.s Based, and St-ilford Loan progrums. 

( J / » lisiiinales are Ironi llic 1988 H9 Inicgraled Quality Control Mcasiircnicnt Projt^ct condiittcd lor the Dcpartnicnl of Education by Price 

a Walerhousc in assot iaiion with Pelavin Associates and The Gallup Organi/.ai m. 
ERIC million .simlcnis awarded Tiile IV iinancial aid during ihc 1*^88-81) award year. 

"M" 3. Based on $15.4 hilliim available to students iluring the I ''88-89 award year, 



lertt 



918 
1,025 
480 



742 
836 
440 



1,125 
1.189 
610 
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Although error rates estimated during IQCMP are lower than were error rates estimated 
during previous quality control studies, there is still a significant amount of error in the 
programs. Almost 11 percent of the Title IV program dollars arc awarded in error. 

B. Error in the Pell Grant Program 

1. Pell Grant Errors 

Error in the Pell Grant Program is defined as the actual Pell award minus the best Pell 
award. For 1988-89 the Pell program distributed $4.9 billion in financial assistance to 
over 3.3 million undergraduate students enrolled in postsecondary institutions. 
Approximately 9.9 percent ($481 million) was awarded in error, and about 28.2 percent 
of students receiving Pell grants had errors of over $50 in their awards. Exhibit II-2 
presents a table of error in Pell awards. 

Other significant findings about Pell Grant Payment Error include: 



• Average absolute enor is $489 for students with error. 

• Student error accounts for sbout $365 million in absolute error. This is 
over two and one-half times the amount of institutional error ($126 
milUon). 

• About 24.8 percent of Pell recipients had errors attributable to students, 
while in only 4.6 percent of cases was there an error attributable to the 
institution. 

• Overawards accounted for a larger share of Pell payment error (7.8 percent 
overall) than did underawards (2.1 percent overall). 
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Exhibit II-2 



Error Measure 



OVERALL 

Absolute 

Overaward 

Undcraward 



STUDENT 

Absolulc 

Overaward 

Underaward 



iNsmunoNAL 

Absolute 

Overaward 

Underaward 



Award Year 1988-89 




Percent of 
Recipients^ 
with Error 



28.2(12.7) 
19.6(±2.2) 
8.6(±2.()) 



24.8(±2.6) 
15.9(12.0) 
8.9(12.0) 



4.6(11.2) 
4.4(11.2) 
0.1(10.2) 



Percent 
of Dollars^ : 
in Error 



..... , 



Amount 
($ Millions) 



9.9(11.4) 
7.8(11.3) 
2.1(10.6) 



7.5(11.1) 
5.4(10.9) 
2.1(10.6) 



2.6(10.9) 
2.5(10.9) 
0.1(10.1) 



481 
379 
102 



365 
263 
102 



126 
122 
5 



With Error 



489 

553 
344 



422 
470 

338 



783 
788 
595 



1, l-sliniaics arc Iroiii llic 1988-89 Inicgralcd Quality Conlrol Mcasurcmcnl Projccl concluded for ihc DcparUncnl of Iiducalion by Price Walcrhouse in associaiion wilh 
Pclavin Ass(Kialcs and The Gallup Orgiinizalion. ^ 0 4 

2. Based on 3.3 million Pell Grams awarded during 1 988-S9 award yi ii! - » 
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2. 



Cost to the Federal Government 



The Pell Grant program cost to the Federal government measures the degree to which 
the intent of the law regarding the Pell Grant program is not being met by the delivery 
system. This measure includes quality errors as well as liabilities. Although it is not 
possible to totally eliminate errors resulting in cost to the Federal government, it is 
desirable to concentrate on reducing these errors. It may be necessary to modify the 
delivery system to meet this goal. 

For the Pell Grant program, the cost to the Federal government is defined to be the Pell 
dollars overawarded. Estimates of the cost to the government due to errors in the Pell 
program are shown in the following table. Note that student and institutional error do 
not sum to overall error in the table because of offsetting errors. 



Pell Program Cost to the Federal Government (Award Year 1988-89) 



Source of Error 



Total Cost 
$ Millions 



Student 



263 



Institutional 



122 



Overall 



379 



ERIC 



II-5 

or 



lOCMP Findings and Corrective Actions 

C Errors in the Campus-Based Programs 

Error in the Campus-Based program is measured in the following three ways: 

• Campus-Based Need Error 

• Campus-Based Distributional Error 

• Campus-Based Award in Excess of Need Error 

1. Campus-Based Need Error 

Student need is defined as the difference between the cost of attendance and the sum of 
the expected family contribution and other known financial aid awarded before Campus- 
Based aid. Campus-Based Need Error is defined as the student need computed from 
reported data minus student need computed from best data. 

Exhibit n-3 summarizes our estimates of Campus-Based Need Error. Approximately 50 
percent of all students receiving Campus-Based aid have errors in calculated need of 
over $50. This amount accounts for only about 11.3 percent of total Campus-Based 
need. The average need enor of students having a need error was $978. Other 
highlights of the analysis of need error include: 

• The percent of institutional absolute need dollars in error (6.2) is very 
close to percent of student need dollars in error (5.7 percent). 

• The percent of students with need error was evenly split between 
overstatement error (26 percent) and understatement error (25 percent). 
However, overstatement error accounted for 7.1 percent of need dollars as 
opposed to only 4.2 percent for understatement error. 
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Exhibit II-3 



CAMPUS-BASED NEED ERROR' 

Award Year 1988-89 







JJiiiiiM; ill. lll'iki 




c-¥ v< x-: x';- : • >■<•■ ■<■■< ••:•-:• •: :->:• 
... • 

...^ • ■ '.^ 


Error Measure 


Percent of 
Recipients^ 

with Rrror 


Percent 
of Dollars^ 
in Krror 


Total 
Amount 
(% Millions) 


With'Error 

($) 


OVERALL 

Absolute 


51.0(±2.9) 


11.3(±1.4) 


656 


978 


Overstatement 


26.0(±2.7) 


7.1 (±1.3) 


412 


1,207 


Understatement 


25.0(±2.5) 


4.2(±0.7) 


244 


740 


STUDENT 










Absolute 


37.6(±2.9) 


5.7(±1.0) 


331 


669 


Overstatement 


21.3(±2.5) 


3.6(±0.8) 


207 


743 


Understatement 


16.3(±2.1) 


2.1 (±0.6) 


122 


572 


INSTITUTIONAL 
Absolute 


20.9(±2.3) 


6.2(±1.1) 


360 


1,307 


Overstatement 


8.6(±1.6) 


3.9(±1.1) 


226 


1,985 


Understatement 


12.2(±1.9) 


2.3(±0.5) 


134 


827 



(17 \. Eslimaies arc from the 1988-89 Integrated Quality ConUol Mcasurcmcni Project conducted for the Department of Education by Price Waterhouse in association with 

^ Pelavin Associates and The Gallup Organization. 

O Based on 2.2 million Campus-Based awards made during 1988-89 year. 

ERJC . Based on an estimated total Campus-Based need of $5.8 billion. ($2.1 billion awarded through Campus-Based programs in 1988-89 award). 
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• The greatest average need error resulted from institutional overstatements 
of need ($1,985). llie smallest average need error ($572) resulted from 
student errors leading to an understatement of need. 

Need error does not translate directly into error in the Campus-Based award because a 
portion of the Campus-Based need is often left unmet. Need error is an important 
indicator of the quality of the Title IV delivery system because consideration of student 
need is an important factor in the financial aid officer's final decision on distribution of 
the institution's Campus-Based funds. 

2. Campus-Based Distributional Error 

Campus-Based Distributional Error, summarized in Exhibit II-4, is an estimate of the 
payment consequences of all need errors. Distributional error is computed as the actual 
Campus-Based award minus the best Campus-Based award, where the best Campus- 
Based award is calcula^'^d using the best need data and the institution's packaging 
guidelines. Distributional error is not an error in the regulatory sense of the word, but 
rather is an indication of the quality of Campus-Based distribution. A distributional 
error could occur when an institution fails to follow its packaging guidelines, even though 
the award did not violate the regulations governing Title IV aid. We did not collect 
documentation sufficient to determine whether departures from an institution's stated 
packaging policies were made intentionally or through oversight. 

In the 1988-89 academic year, over $2.1 billion was awarded to students participating in 
the Campus-Based programs. Of this amount 21.5 percent, or $453 million, was awarded 
in error. About 43 percent of students receiving Campus-Based aid have absolute errors 
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Exhibit n-4 



CAMPUS-BASED DISTRIBUTIONAL ERROR' 

Award Year 1988-89 



Error Measure 




OVERALL 

A KerklllfP 


43 0 (±2 9) 


21 5 (±2 3^ 


453 


819 


Overaward 


32.7 (±2.8) 


17.3 (±2.1) 


364 


864 


Underaward 


10.3 (±1.8) 


4.3 (±1.2) 


91 


* 876 


STUDENT 

Absolute 


30.9 (±2.8) 


15.1 (±2.0) 


318 


799 


Overaward 


23.9 (±2.6) 


12.3 (±1.8) 


259 


843 


Underaward 


7.0 (±1.5) 


2.8 (±1.0) 


59 


649 


INSTITUTIONAL 
Absolute 


18.5 (±2.2) 


9.0 (±1.5) 


190 


801 


Overaward 


13.7 (±2.0) 


7.2 (±1.4) 


152 


866 


Underaward 


4.8 (±1.2) 


1.8 (±0.7) 


38 


615 
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1. Estimates arc from the 1988-89 Integrated Quality Control Measurement Project conducted for the Dcpariincni of Education by Price Watcrhouse in association with 
Pclaviii Associates and The Gallup Organization. 
FR?r ^- ^^^'^ '^'^ million Campus-Based awirds made during 1988-89 award year. 101 
u& 3. Based on $2.1 billion awarded through die Campus-Based programs in 1988-8^' 



-89 award year. 



inrMP Findings and Corrective Actions 

in award of over $50. Other significant results of the analysis of Campus-Based 
Distributional Error include: 

• While absolute student errors are more frequent than absolute institutional 
errors in Campus-Based distribution (30o9 percent of students versus 18.5 
percent, respectively), the dollar impact of Campus-Based Distributional 
error institutions is similar ($799 for absolute student error and $801 for 
absolute institutional error). 

• Overawards account for a much greater percentage of dollar errors (17.3 
percent) than do underawards (4.3 percent). 

Distributional error is important because this type of error measures the amount of 
Campus-Based dollars (21 percent overall) that would be distributed differently, either by 
awarding aid to other students or by changing the amounts awarded to the same 
student,, if there were no errors in the delivery system. 

3. Campus-Based Award in Excess of Need Error 

Campus-Based award m excess of need, a measure that approximates the regulatory 
definition of error in the Campus-Based program, is the amount by which the actual 
Campus-Based award exceeds the best Campus-Based need. Award in excess of need, as 
shown in Exhibit 11-5, totalled $67 million in the 1988-89 award year, or about 3.2 
percent of all dollars awarded. The mean error per recipient was $629 for the 8.3 
percent of all students who received Campus-Based financial assistance and who had an 
award in excess of need. About 7.4 percent of recipients had student errors, amounting 
to a total of $57 million in error. Only 3.8 percent of recipients had Campus-Based 
awards in excess of need attributable to institutional error, representing $36 million. 
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Exhibit II-5 



CAMPUS-BASED AWARD IN EXCESS OF NEED' 





Award Year 1988-89 
















Error Mensiir6 


Percent of \ 
Recipients? 

with Error 


Percent 
of Dollars^ 
in Error 


: • Total 
Amount 

($Mimotis) 




Overall 


8.3(±K9) 


3.2{±0.S) 


67 


629 


Student 






57 


589 


Institutional 


3.8(±1.2) 


1.7(±0.6) 


36 


714 



p 1. Eslimatcs arc from ihc 1988-89 Inlegralcd Quality Conlrol Mcasurcmcnl Projccl conduclcd for ihc Dcparuncni of Educallon by Price Walerhouse in associaiion wilh 
6 Fclavin AssiKiaics and The Gallup Organization. 
ERJC 2. Based on 2.2 million Campus-Based awards made during ihc 1988-89 awiud yair. 1 n 1 
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4. Cost to Federal Goveniment 

Unlike the Pell program, Campus-Based errors are not directly tied to Federal 
government costs for the following two reasons: 

• Only a portion of the money distributed in the Campus-Based programs is 
paid by the government. 

• A financial aid administrator has considerable freedom in setting policies 
for distributing Campus-Based program funds at his or her institution. An 
error corrected for one student could mean a redistribution of funds to 
other students, instead of a savings to the Federal government (e.g., if the 
institution had discovered a $500 overaward to a student, the money might 
have been awarded to another student instead of being returned to the 
government). 

The cost to the Federal government due to errors in the Campus-Based program was 
estimated by multiplying the total amount of Campus-Based awards in excess of need 
($67 million) by the percentage of aU Campus-Based dollars that were contributed 
through Title IV funds (60.3 percent). The 603 percent figure was calculated as a 
weighted average of percentage costs for the Perkins Loan, Campus Work Study, and 
SEOG programs as detailed in the table below. The second table on the following page 
shows cost to the Federal government by source of en or. 

D. Error in the StafTord Loan Program 
Error in the Stafford Loan Program is measured by certification error and loan amount 
error. 

L Stafford Loan Overcertification Error 
Stafford Loan Overcertification Error is computed as the actual certification minus the 
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Campus-Based Cost to tlie Federal Government (Award Year 198S*89) 



Campus-Based 
Program 



Campus-Based 
Available Aid 
(% millions^. 



Federal Govermnent 
Cost as a Percentage 
of Aid Available 



Perkins Loan 884.0 


23.2 1 


College Work Study 7803 


80.0 


SEOG 442.4 


100.0 


Weighted Average 


60.3 




Campus-Based Cost to the Federal Government (Award Year 1988-89) 1 


Source of Error 


Total Cost 1 




$ Millions 


Student 


34 


Institutional 


22 


Overall 


40 
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best certification. For the purposes of this study, undercertifications are not considered 
to be errors, as some students purposely accept a smaller loan. The best certification is 
the best Stafford Loan need or the program limit, whichever is less; the best Stafford 
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P Findings and Corrective Actions 



Loan need is calculated as the difference between the cost of attendance and the sum of 
the expected family contribution and other financial assistance available at the time of 
Stafford award was calculated. Exhibit II-6 presents a table of certification errors in the 
Stafford Loan program. Highlights of this analysis include: 



• Stafford Loan overcertificauon errors affect 9.6 percent of all certification 
dollars and 203 percent of the students who are certified for Stafford 
Loans. The mean error per student with overcertification error is $l,277v 

• The percent of certification dollars in error is higher for institutional erroi's 
than for student errors (6.0 percent of dollars for institutions versus 4.2 
percent of dollars for students). However, student errors were found for 
12.9 percent of all certifications while institutional errors were observed for 
only 11.1 percent of the certifications. 



Stafford Loan Overaward Enor is defined as the extent to wliich the actual Stafford 
Loan exceeds the best certification amount. 

The 1988-89 Stafford Loan program made $8.4 billion in aid available to 3.2 million 
students attending postsecondary institutions. Of this amount, 18.3 percent of all loans 
contained an overaward error. This translated to over $742 million in overaward error, 
or 8.8 percent of all dollars awarded. Exhibit II-7 presents a summary of Stafford Loan 
Overaward Errors. Other findings of this analysis include: 

• Student enors accounted for 4.0 percent of the loan dollars, or $337 
million. The average smdent error for students having error was $880. 

• Institutional errors amounted to $447 million, or 5.3 percent of the total 

Srogram dollars. The average Stafford overaward institutional error was 
1,390 for those recipients having error. 
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Stafford Loan Overaward Error 
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Exhibit II-6 



STAFFORD LOAN OVERCERTIFICATION ERROR* 

Award Year 1988-89 



9 













Error Measure.' 


.■■:J-gPerccntpf'>/" 






■ 


Overall 


20.3(±1.8) 


9.6(±1.7) 


860 


1,277 


Student 


12.9(±2.9) 


4.2(±1.0) 


376 


870 


Instilutional 

ICS 


11. 1 (±2.7) 


6.0(±1.4) 


537 


1,464 



1. Eslimalcs are from Ihe 1988-89 Inlegraled Quality Control Measurement Project conducted for the Department of Education by Price Watcrhousc in association with 

O Pclavin Associates and The Gallup Organization. 1 C *• ^ 

ERIC Based on 3.2 million Stafford Loans awarded in the 1988-89 award year. Estimated total certifications equaled $9.0 billion. 

'—--3. Based on $8.4 billion available to students Ihroiich Stafford I. onn nropram diirinp 1988-89 award vnar. 



Exhibit II-7 



STAFFORD LOAN OVERAWARD ERROR* 

Award Year 1988-89 




Error Measure 



, . . . ♦ v.y. ^ 
, •**.• V *., : ' . 



Percent ofl 
]Recipient$^: 
with Error 



•..■...*.::-A.v'.:- 

.... v.v.*. w,'.'.'.'.:;' 

.•.•.•.•.•.■.•.•.•.■.•.■.■.•.•.v.v.y.v." 



.^-..._•L_^A^•:::••::>^^^^ 



itt Error 



%V>A'.v.v.vy.;.:.:.:.w 

A . • w.'Ky ... A', /.y 
. . .V. % .V. A V 

^ • . . . • ,•, •. 
v.-.w.v.v.v. •.•.•.■.•••.•.•.« v.v 




($ Millioiis) 



Overall 



«— • 



Student 



18.3(±2.8) 



8.8(±1.6) 



11.5(±2.7) 



4.0(±1.1) 



742 



337 



1,223 



880 



Institutional 



10 



9.6(±1.6) 



5.3(±1.3) 



447 



1,390 



1, Eslimalcs arc from the 1988-89 Irccgralcd Quality Control Measurement Project conducted for the Department of Education by Price Waterhousc in association with 
P R?r Pclavin Associates and The Gallup Organization. 
SML 2. Based on 3.2 million Stafford Loans awarded during 1988-89 award year. 
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lOCMP Findinp and Corrective Actional 

Much of the Stafford Overaward Error is attributable to other resources made available 
to the student after the Stafford Loan has been certified. While this is not considered a 
regulatory error, it is a quality error measured by IQCMP and does lead to additional 
cost to the Federal government. 

3. Cost to the Federal Government 

As with Campus-Based error, care should be taken not to interpret Stafford Loan error 
as identical to cost to the Federal government. Also, these overaward figures include 
quality errors and are not necessarily liabilities. Nonetheless, a significant problem exists 
as shown in these estimates. 

To estimate the cost to the Federal government for the Stafford Loan program over the 
life, the total overaward dollars were multiplied by 0.513, the program costs per dollar 
loaned, as estimated by ED. Appendix C presents a worksheet used by ED to calculate 
the program costs per dollar loaned. Estimates for the cost to the Federal government 
due to Stafford Loan errors for the 1988-89 award year are shown in the table below. 

E. Comparison of IQCMP and Stage II Results 

This section compares key error measurements from the current IQCMP study 
(evaluating the 1988-89 award year) with those of the prior Stage Two study (evaluating 
the 1985-86 award year). These results are summarized in Exhibit II-8. 

As shown, current study estimates of Pell and Campus-Based error are significantly 
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Stafford Loan Cost to the Federal Government (Award Year 1988-89) 

Source of Error Total Life Cycle' 
Cost $ Millions 

Student 172 

Institutional 229 

Overall 380 

Assumes a 10 year repayment period. 



smaller than those of Stage Two. Differences between the two sets of results are most 
noticeable for Pell institutional error and Campus-Based student error. The two studies 
provide similar projections for Stafford ovcrcertification error. 

Differences between the two sets of error estimates are due to two factors: (1) changes 
to the financial aid delivery system, and (2) differences between the two study 
methodologies. 

Several changes have been made to the financial aid deUvery system during the 3-year 
period between studies. Some of these changes have undoubtedly contributed to a true 
reduction in error rates. For example, the fi'^ancial aid formula has been revised to 

n-is 



ERIC 



Exhibit n-8 



ABSOLUTE ERROR RATE COMPARISON: 
IQCMP (1988-1989) AND STAGE TWO (1985-1986) 



PELL ERROR 
Overall 

Student 

Institutional 



CAMPUS-BASED NEED 
ERROR 

Overall 
Student 
Institutional 




Error Measure 



CAMPUS-BASED AWARD 
IN EXCESS OF NEED 

Overall 
Student 
Institutional 



IQCMP 



28.2 
24.8 
4.6 



54.4 
32.3 
30.0 



51.0 
37.6 
20.9 



77.2 
64.5 
31.5 



8.3 
7.4 
3.8 



22.5 
12.7 
13.4 



y s^v. s^ wM. "^ft: ^1^*1^211 



9.9 
7.5 
2.6 



21.1 
12.0 
11.0 



11.3 
5.7 
6.2 



23.8 
18.6 
7.9 



3.2 
2.7 
1.7 



14.0 
7.2 
7.2 



IQCMP 



STAGE 
TWO 



489 
422 
783 



978 
669 
1.307 



629 
589 
714 



502 
486 
460 



1,080 
1,012 
877 



921 
844 
793 



CAMPUS-BASED 
DISTRIBUTIONAL ERROR 

Overall 
Student 
Institutional 



43.0 


69.1 


21.5 


30.2 


819 


649 


30.9 


60.5 


15.1 


25.0 


799 


613 


18.5 


27.3 


9.0 


10.9 


801 


590 



STAFFORD LOAN 
OVERCERTIHCATION 

Overall 
Student 
Institutional 



20.3 
12.9 
11.1 



20.1 
10.6 
13.5 



9.6 
4.2 
6.0 



10.2 
4.4 
6.5 



1,277 1,306 
870 1,065 
1,464 1,238 
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lOCMP Findings and Corrective Actions 

reduce or eliminate the effect of certain error-prone data elements (e.g., the current 
formula no longer uses expected year income in place of base year income for certain 
dependent students). Also, the use of professional judgment to override individual data 
elements, Pell Student Aid Indexes, and award amounts was introduced in the Pell and 
Stafford Loan programs. The increased use of professional judgment will decrease error 
rates, because the changed data is never considered to be an error, provided the basis of 
the professional judgment is documented. Although there have been significant ch^Jiges 
to the jOnancial aid delivery system, differences between the methodologies used in the 
two quality measurement studies are significant enough to make uninformed comparisons 
between the two sets of study results misleading. 

The Stage Two smdy was ED's first comprehensive quality measurement study of the 
Title IV programs. While the current IQCMP is largely patterned after the Stage Two 
study, certain methodological enhancements were incorporated in ED's second 
comprehensive quality measurement study. Many of these enhancements are extensions 
of approaches used in the Stage Two smdy that had been suggested by Stage Two project 
team members. 

Each methodological difference between IQCMP and Stage Two was carefully examined 
prior to implementation. Analysis of IQCMP and Stage Two data confirmed that the 
changes greatiy improve the precision of the error measurements. As further 
confirmatory evidence, we compared IQCMP results for institutions participating in the 
Institutional Quality Control Pilot (IQC Pilot) project to error measurements made 
independently by the IQC Pilot schools themselves. The similarity of our measurements 
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to those made by the IQC Pilot schools supported our conclusion that the 
methodological changes resulted in precise error measurements. 

Our analysis of IQCMP and Stage Two data showed that the enhancements included in 
IQCMP generally reduce, often significantly, the level of error estimated by the study. 
Thus, much of the difference between IQCMP and Stage Two results is probably 
attributable to the improved measurement of delivery system error. Unfortunately, we 
are unable to quantify the extent to which differences in the results from the two studies 
reflect true reductions in delivery system error. 

Even though the IQCMP estimates of delivery system error are smaller than the Stage 
Two estimate, the IQCMP results still show that there is a significant amount of error in 
the Title IV delivery system. This ntmaining error may be reduced through further 
improvements in the procedures ana methods used in awarding student financial 
assistance. Our recommended actions tj reduce error are contained in Section VII of 
this report. 
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lOCMP Findings and Corrective Actions 

SOURCES OF ERROR IN THE TITLE IV FINANCIAL AID PROGRAMS 



A. Overview 

Chapter n discussed ovei'all institutional and student error in the Title IV programs. 
This chapter examines the components of institutional and student error. Institutional 
and student enor in each of the three programs is further decomposed into marginal 
error, the most detailed level of error in this study. Marginal errors measure the effect 
of errors caused by individual components of award calculation and disbursement. 

Marginal error is calculated by comparing: (1) the award calculated with all reported 
values, and (2) the award calculated with all reported values and one "best" value. The 
difference between the two awards represents the marginal error caused by substituting 
the best item. Like overall error, marginal error can be divided into institutional 
marginal error and student marginal error. Although each marginal error represents a 
component of overall error, marginal error will not sum to overall institutional and 
student error because: 

• some errors may cancel other errors 

• some errors tend to occur in combination with other errors 

• some errors have a carryover effect on other errors (i.e., by being in error 
they will, by definition, cause other items to be in error) 

Only those items which produced significant marginal error are discussed in this chapter. 
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B. Student Marginal Error 

Student marginai error refers to the error caused by the student misreporting an 
individual application item. This type of error occurs when there is a significant 
discrepancy between the student reported value and the "best" value for a given 
application item. Best values for application items were obtained from sources outside 
the institution (e.g., tax return, parent interview) whenever possible. Student m arginal 
error is calculated by substituting the reported value for a single application item (e.g.. 
Student's Adjusted Gross Income) with its "best" value. A new SAI and FC are then 
calculated using the "best" Student Adjusted Gross Income, and these are used to 
recompute the best awards and certifications. The difference between the award 
calculated with the substituted 'Tjest" value and the award calculated using only reported 
values represents the marginal error associated with the substituted application item. 

Marginal errors are calculated independently for each application item, and may not 
necessarily sum to overall student error. Therefore, a marginal error on the student's 
part should not be viewed as the amount of error reduction attributable to perfect 
reporting of that data item, but rather the relative contribution to error of that item 
assuming no corrections to other incorrectly reported items. 

Student marginal errors were calculated for three subpopulations: dependent students, 
independent students, and all students. The subpopulation used was determined by the 
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lOCMP Findings and Corrective Actions 

group of students that had a value for that application item. For example, the marginal 
error for Student Adjusted Gross Income was calculated for all students, but marginal 
error for Parent Adjusted Gross Income was calculated for dependent students only. 
Similarly, marginal error for Student Number in College was calculated for independent 
students only. Exhibit III-l lists the individual application items tested for marginal error 
and the subpopulation of students on which the error was calculated (dependent 
students, independent students, or both). 

All student marginal errors were calculated using a $50 tolerance, and measured for 
their effect on Pell absolute award, Campus-Based need, and Sta£Eord Loan 
overcertiflcation. Exhibit 111-2 presents average student marginal error across programs. 
Only those items which are provided significant marginal error are discussed m this 
section. Appendix A provides a complete listing of each item's marginal contribution to 
error. Examination of the significant smdent marginal errors led to the following 
findings: 

• In general, student marginal error is greater for the variables reported only 
by dependent students than it is for the variables reported only by 
independent smdents or for the variables reported by all students. 

• Student Adjusted Gross Income and Student Cash/Savings/Checking 
account for the greatest number of errors among the variables reported by 
all smdents. Student Adjusted Gross Income accounts for the greatest 
percent of dollars in error. Errors in Student Cash/Savings/Checking, 
while frequent, generally had a small effect on the dollar amount awarded. 
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E?^hibitni-1 
Page i of 5 

SIIBPOPULATTON FOR MAROTNAL STUDE NT ERROR TTRMS 




Date Of Birth 


All Students 


U.S. Aimed Forces Veteran 


All Students 


Waid Of Court 


All Students 


Legal Dependent Other Than Spouse 


All Students 


Claimed As Exemptions On Parents 1986 
Income Tax Return 


All Students 


Claimed As Exemptions On Parents 1987 
Income Tax Return 


A ft J 

All Students 


Clauned As Exemptions On Parents 198o 
Income Tax Return 


All Students 


Received Federal Student Aid 1987-88 


All Students 


1985 Total Resources Of $4,000 Or More 


All Students 


1986 Total Resources Of $4,000 Or More 


All Students 


1987 Total Resources Of $4,000 Or More 


All oiuuenis 


U.S. Citizen 


All Students 


Year In College 


All Students 


Candidate for a Bachelor's Degree 


All Students 


Marital Status 


All Students 


Student's (& spouse's) 1987 Filing StaUis 


All Students 


Student's (& spouse's) 1987 Exemptions 


All Students 


Student's (Adjusted Gross) 1987 Income 


All Students 
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SIIBPOPITT ATTON FOR MARGINAL STUDENT ERROR ITEMS 







Student's (& spouse's) 1987 Taxes Paid 


All Students 




All Students 


Student's 198/ Income From Work 


All Students 


Spouse's 1987 iticome From Woric 


All Students 


Student's (& spouse's) 1987 Social Security 
Benefits 


All Students 


Student's (& spouse's) 1987 AFDC Or ADC 


All Smdents 


Student's (& spouse's) 1987 Child Support 
Received 


All Students 


Student's (& spouse's) 1987 Other Untaxed 
income /vno oeneiiis 


All Students 


Oulueni S \0L SpUUSC oj L/lSlUCalCU VYUlKCr OUtlUd 


All Students 


Student s (& spouse s) Displaced rlomemaRer 
Status 


All SfiiHpnt^ 


Student's (& spouse's) Cash, Checking, Savings 


Alt Students 


^tiiHpnt'c (Xr cnniicp**c^ Hnirip Value 


All Students 


oiuuciii s ^oc spuusc o) OUIIlC UCUl 


All Students 


ouiQent s \6L spouse uuier KCai c^siaic/invcsuucni 
Value 


All Students 


Student's (& soouse's) Other Real Estate Investment 
Debt 


All Students 


Student's (& spouse's) Business And Farm Value 


All Students 


Student's (& spouse's) Business And Farm Debt 


All Students 


Student's Expected GI Bill And Dependents' 
Educational Assistance Per Month 


All Students 
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SlIBPOPin.ATinN FOR MARO TNAI. STIIPFNT ERROR ITEMS 



••••■>: : '.v ^ ■ '>■" . ■ ' - : :'t/ . • . ■ - - . 
1: •'■ ' ^ ' ' . ^' ' ' ' * ' . 




Student's Expected GI Bill And Dependents' 
Educational Assistance Benefits Number Of Months 


All Students 


Smdent's Expected 1988 Income From Work 


All Students 


Spouse's Expected 1988 Income From Work 


All Students 


Student's (& spouse's) Expected 1988 Other 
Taxable Income 


All Students 


Studeni's (& spouse's) Expected 1988 Untaxed 
Income And Benefits 


All Students 


Parents' Marital Status 

A ^mm^0mmmi%9 JkvA%<M*w«wA h^k>%i%V«MM 


Denendent Students 


Parents' State Of Legal Residence 


Dependent Smdents 


Parents' 1987 Filing Status 


E>ependent Students 


Parents' 1987 Exemptions 


Dependent Students 


Parents' 1987 Income 


Dependent Students 


Parents' 1987 Taxes Paid 


Dependent Students 


Parents' 1987 Itemized Deductions 


Dependent Students 


Fathers' 1987 Income From Work 


Dependent Students 


Mothers' 1987 Income From Work 


Dependent Students 


Parents' 1987 Social Security Benefits 


Dependent Students 


Parents' 1987 AFDC or ADC 


Dependent Students 


Parents' 1987 Child Support Received 


Dependent Students 


Parents' 1987 Other Untaxed Income And Benefits 


Dependent Students 


Parents' 1987 Total Untaxed Income And Benefits 


Dependent Students 


Student's (& spouse's) 1987 Touil Untaxed Income 
And Benefits 


Dependent Students 
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Parents' Home Debt 


Dependent Students 


Parents* Other Real Estate/Investment Value 


Dependent Students 


Parents' Other Real Estate/Investment Debt 


ucpeiiQeni oiUQcnis 


A aroniS I3UolIlwoo /viiu jranii vaiuw 


uependent students 


Parents Business And Farm Debt 


Dependent Students 


Parents 1987 Medical And Dental Expenses 


ucpcnucni oiuucnio 


Parents' 1987 Education Expenses 


uepenaeni ouiaenis 


Parents' 1987 Children With Education Expenses 


£)ependent Students 


Parents' Dislocated Worker Status 


Dependent Students 


Parents' Displaced Homemaker Status 


Dependent Students 


Age Of Older Parent 


Dependent Students 


Parents' Cash, Checking, Savings 


Dependent Students 


Parents' Home Value 


Dependent Students 


Parents' Household Size 


Dependent Students 
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SITRPOPIILATinN FOR MARGINAL STUD FNT FRROR ITEMS 



** • 

, . . ..• .•, y-.j^.:A-.-:\-i^:^.-^.-.-.\-.^^^^^ 




Parents' Number In College 


Dependent Students 


Student s Number In College 


Dependent otudents 


ramers cxpectea i7oo mcome rroni wofk 




Mother's Expected 1988 Income From Work 


Dependent Students 


Parents' Expected 1988 Other Taxable Income 


Dependent Students 


Parents Expected 1988 Untaxed Income And 
Benefits 


uepenaeni ouiaenis 


Student's (& spouse's) 1987 Medical And Dental 
Expenses 


Independent Students 


Student's (& spouse's) 1987 Education Expenses 


Independent Students 


Student's (& spouse's) 1987 Children With Education 
Expenses 


Independent Students 


Student's Household Size 


Independent Students 


Student's Veterans Contributory Benefits Per Month 


Independent Students 


Student's Veterans Contributory Benefits Number 
Of Months 


Independent Students 
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Exhibit ni.2 

SIGNIFICANT STUDENT MARGINAL ERROR 



Average Error Across Programs 
Award Year 1988 - 89 









^^^^^^^ 


Error Type 


Percent of 
Recipients 
with Error 


of Dollars 
in Horror 


Mean Error 
Per Recipient 
with Error ($) 


ALL STUDENTS 
Student Adjusted Gross Income 


2.5 


0.8 


888 


Student Cash/ Savings/ Checking 


2.8 


0.3 


225 


Student Work Income 


1.9 


0.3 


457 


Student Other Untaxed Income 


1.7 


0.4 


707 


Student U.S. Taxes 


1.2 


0.1 


183 


DEPENDENT STUDENTS ONLY 








Parent Household Size 


10.5 


2.2 


529 


Parent Home Value 


6.1 


1.4 


491 


Parent Other Untaxed Income 


5.2 


1.4 


398 


Parent Number in College 


4.9 


1.5 


834 


Parent Real Estate/Investment 
Value 


2.6 


0.9 


563 


Parent Work Income 


2.3 


0.2 


270 


Parent Adjusted Gross Income 


2.2 


0.9 


1,023 


Parent U.S. Taxes 


2.2 


0.3 


406 


Parent's Spouse Work Income 


2.2 


0.3 


267 


Parent's Home Debt 


2.2 


0.2 


271 


INDEPENDENT STUDENTS ONLY 
Student Household Size 
Student Number in College 


2.3 
1.7 


1.1 
0.4 


2,296 
311 



' Unweighted Average of: (1) Pell Absolute Award Error, (2) Campus-Based Need Error, and (3) Stafford Overcertification Error. 
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• Among the variables reported by dependent students, Parent Household 
Size generally accounts for the greatest number of errors. Errors in 
Parents Adjusted Gross Income, while not the most frequent among 
dependent students, contribute the largest amount of error per dependent 
student with error ($1,023 per dependent student). 

• Student Household Size and Smdent Number in College are the only 
significant contributors to error among those variables reported by only 
independent students. Average error per independent student with a 
Student Household Size error was greatest at $2,296. 

1. Student Marginal Error in the Pell Program 

As reported in Chapter II, absolute student error for Pell Grant Payment was found in 
24.8 percent of all Pell students in the study. These errors account for 7.5 percent of the 
dollars in error with u mean error of $422 per student with error. Student marginal 
error m the Pell Grant Program further decomposes these enor rates and attributes the 
overall error to individual application items. 

Exhibit III-3 presents the most significant student marginal errors in the Pell Program, 
the percent of students that had each error, and the effect each type of error had on 
award dollars. All figures represent absolute error since the low number of students in 
each error category make estimates of overaward and underaward errors unreliable. 
Examining student marginal error in the Pell Grant Progiam produced the following 
findings: 
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SIGNIFICANT STUDENT MARGINAL ERROR 

Pell Absolute Error 
Award Year 1988-89 







liiiTor type 


Percent of 
Recipients 
with Error 


Percent 
of Dollars 
in Error 


Mean Error 
Per Recipient 
witli Error ($) 


ALL STUDENTS 








Student Cash/ Savings/ Checking 


3.1 


0.5 


205 


Student Adjusted Gross Income 


1.6 


0.9 


786 


Student Other Untaxed Income 


1.5 


0.5 


476 


Student U.S. Taxes 


0.9 


0.1 


139 


Student Work Income 


0.7 


0.2 


290 


Student Real Estate/ Investment 
Value 


0.5 


0.3 


729 


DEPENDENT STUDENTS ONLY 








Parent Household Size 


15.5 


3.4 


283 


Parent Other Untaxed Income 


10.3 


3.6 


448 


Porent Home V^uc 


6.6 


2.3 


444 


Parent Social Securitv Benefits 






529 


Parent Number in College 


3.7 


0.6 




Parent U.S. Taxes 


1.9 


0.4 




Parent Real Estate/ Investment 
Value 


1.5 


1.7 


611 


Parent Adjusted Gross Income 


1.4 


0.3 


249 


Parent Work Income 


1.4 


0.3 


290 


Parent's Spouse Work Income 


1.2 


0.3 


296 


INDEPENDENT STUDENTS ONLY 








Student Household Size 


2.9 


1.1 


576 


Student Number in College 


1.4 


0.5 


521 
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• Among the variables reported by all students, errors in Student 
Cash/Savings/Cheddng are the most frequent (3.1 percent of all Veil 
students) while errors in Student Adjusted Gross Income are the most 
costly (0.9 percent of dollars in error and mean error of $786 per recipient 
with error). 

• Among the variables reported by dependent students, Parent Household 
Size and Other Untaxed Income contribute most to Pell error (15.5 percent 
and 10.3 percent of the recipients and 3.4 percent and 3.6 percent of the 
dollars, respectively). 

• For independent students. Student Household Size accounts for roughly 
twice as many errors as Student Number in College (2.9 percent versus 1.4 
percent) and has nearly three times the impact on dollars in error (1.1 
percent versus 0.4 percent). 

• Although not the most frequently erroneous item, errors in Real 
Estate/Investment Value for both parent and student make significant 
contributions to dollars in error (mean errors per student with error of 
$611 and $729 respectively). 



2* Student Marginal Error in the Campus-Based Programs 



Absolute student need error in the Campus-Based program was identified in 37.6 percent 
of all Campus-Based recipients (Exhibit II-3 of the prior chapter). The effect on total 
dollars in error was measured at 5.7 percent, with a mean error of $669 for recipients 
with error. Like Pell Payment error, Campus-Based need error may be further 
decomposed to identify the source of error through marginal error analysis. 



Exhibit III-4 presents the most significant student marginal errors in Campus-Based need 
error, the percentage of Campus-Based students with the identified error, and the 
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SIGNIFICANT STUDENT MARGINAL ERROR 

Campus Based Need Error 
Award Year 1988 - 89 




Error Type 



ALL STUDENTS 
Student Cash/ Savings/ Checking 

Student Adjusted Gross Income 

Student Work Income 

Student Other Untaxed Income 

Student U.S. Taxes 

Student Real Estate/ Investment 
Value 

Student's Spouse Work Income 
Student Social Security Benefits 



DEPENDENT STUDENTS ONLY 
Parent Household Size 

Parent Number in College 

Parent Home Value 

Parent Work Income 

Parent Real Estate/Investment 
Value 

Parent Adjusted Gross Income 
Parent U.S. Taxes 
Parent's Spouse Work Income 
Parent Other Untaxed Income 



INDEPENDENT STUDENTS ONLY 
Student Number in College 

Student Household Size 



Percent of 
Recipients 
with Error 



4.3 
3.8 
3.6 
2.2 
1.9 
1.2 

0.6 
0.4 



10.6 

8.3 

7.7 

4.3 

4.0 

3.7 
3.6 
3.2 
1.8 



3.5 
3.4 



LESS THAN 0.05% 



Percent 
of Dollars 
in Error 



0.3 
0.5 
0.3 
0.4 
0.1 
0.2 

0.1 



1.9 
2.6 
1.0 
0.3 
0.4 

1.1 
0.2 
0.2 
0.2 



0.2 
1.2 



Mean Error 
Per Recipient 
with Error ($) 



299 
645 

335 
898 
225 
659 

329 
739 



705 
1,238 
489 
256 
396 

1,145 
230 
237 
435 



261 
1,850 
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contribution to total dollars in error. Again, all figures represent absolute error and . 
were calculated using a $50 tolerance on the error. Significant findings of the Campus- 
Based marginal error analysis include: 

• Amoiig the variables reported by all students, Student Cash/Savings/ 
Checking accounts for the greatest number of Campus-Based students in 
error (43 percent) while Student Adjusted Gross Income contributes most 
to Campus-Based dollar error (0.5 percent). 

• Parent Nimiber in College and Parent Adjusted Gross Income are the most 
frequent among those variables reported only by dependent smdents and 
cause large dollar errors when these items are missed (mean error per 
smdent with error of $1,238 and $1,145, respectively). 

• Among independent students, errors in Student Household Size and 
Student Number in College affect roughly the same number of smdents 
(3.4 percent and 3.5 percent respectively). While Student Household Size 
contributes much more than Student Nimiber in College to total dollars in 
error (1.2 percent versus 0.2 percent), the small number ol errors observed 
in our sample may make this measurement unreUable. 



3. Student Marginal Error in the Stafford Loan Program 



Stafford Loan student overcertification error was found in 12.9 percent of students who 
received a Stafford Loan. This translated to 4.2 percent of the total dollars in error with 
an average error of $870 for those students with error. Decomposing this overall error 
through marginal error analysis isolates those application items prone to error in the 
Stafford Loan certification process. 

Exhibit III-5 presents the most significant student marginal errors in certifying a Stafford 
Loan, the percent of Stafford Loan recipients affected, and the contribution to dollar 
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Exhibit Ul-S 



SIGNTFICANT STUDENT MARGINAL ERROR 

Stafford Loan Overcertification Error 
Award Year 1988 • 89 











Error Type 


Percent of 
Recipients 
with Error 


Percent 
of Dollars 
in Error 


Mean Error 
Per Recipient 
with Error ($) 


ALL STUDENTS 

Student Adjusted Gross Income 


2.2 


1 n 


1,234 


Student Other Untaxed Income 


L5 


0.4 


748 


Student Work Income 


L4 


0.4 


747 


Student U.S. Taxes 


LO 


0.1 


184 


Student Cash/ Savings/ Checking 


LO 


0.1 


170 


Student Real Estate/ Investment 
Value 


0.4 


0.2 


1,125 


Student's Spouse Work Income 






ISA 
loo 


DEPENDENT STUDENTS ONLY 
Parent Household Size 


5.4 


1.4 


600 


Parent Home Value 


4.0 


1.0 


541 


Parent Other Untaxed Income 


3.7 


0.5 


313 


Parent Number in College 


2.6 


1.2 


1,063 


Parent Social Security Benefits 


2.6 


0.7 


613 


Parent Real Estate/ Investment 
Value 


2.4 


0.7 


682 


Parent U.S. Taxes 


LO 


0.3 


741 


Parent Adjusted Gross Income 


1.6 


1.2 


1,675 


Parent Work Income 


1.2 


0.1 


264 


INDEPENDENT STUDENTS ONLY 








Student Household Size 


0.7 


1.0 


4,462 


Student Number in College 


0.2 


* 


152 


♦LESS THAN 0.05% 
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error associated with each application item. An examination of the student marginal 
errors affecting Stafford Loan certification produced the following results: 



• In general, there are fewer students with Stafford Loan certification error 
than with Campus-Based need or Pell award error, but the mean error for 
students with Stafford overcertification error is, on average, much higher. 
This results because the typical Stafford certification is much larger than 
the typical Pell award and generally higher than the Campus-Based need. 

• Among the variables reported by all students. Student Adjusted Gross 
Income is by far the largest contributor to Stafford overcertification enor 
(2.2 percent of Stafford Loan recipients in error account for 1.0 percent of 
total dollars in error). 

• Parent Home Value is ranked just below Parent Household Size for 
contribution to error among dependent smdents. Errors in Parent 
Household Size affected S.4 percent of the Stafford Loan students, while 
Parent Home Value affected 4.0 percent 

• Among dependent students, Parent Adjusted Gross Income accounts for 
only L6 percent of the recipients with error, but recipients had an average 
overcertification error of $1,675. Parent Number in College caused an 
average overcertification error of $1,063 per student with error. 

• Among independent students receiving Stafford Loans, only 0.7 percent 
h^d an error in Student Household Size and only 0.2 percent had an error 
in Student Number in College. 



4« Student Marginal Error Groupings 

One of the limitations of marginal error is that, because usually a small percentage 
students have an error for a single appUcation item, estimates of marginal error may be 
unreUable. Grouping application items, however, allows more precise evaluations of 
error. Exhibit III-6 lists the logical groupings of student c ata items for which we 
measured student error. Grouped marginal errors are calculated by substituting the 
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RVPLANATION OF STU DENT MARGINAL ERROR GROUPS 



Group: 



Subpqpulation 



AppDcatton 
Items Included 
In Group 



Dependency 
Status Items 



All Students 



•Student DateofBiith 
•Number of legal 
dependents 
•Ward of the Court 

• U.S. Armed Forces 
Veteran 

• Student Marital 
Status 

• Year in college 

• Federal aid received 
1987-88 

• 1985 total resources 
of $4000 or more 

• 1986 total resources 
of $4000 or more 

• 1987 total resources 
of $4000 or more 

•Claimed as 
exemption on parents 

1986 tax return 

• Claimed as 
exemption on parents 

1987 lax return 

• Claimed as 
exemption on parents 

1988 tax return 



Student Tax * 
Items 
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All Students 



Student 
Untaxed Items 



All Students 



Student adjusted 
gross income 
Student U.S. 
taxes paid 
Student work 
income 
Student*s spouse 
work income 
Student iiemizec 
deductions 
Student tax form 
filed 

Student number 
of exemptions 



Student child 
support received 
Student Social 
Security benefits 
Student AFDC 
received 
Student other 
untaxed income 



Student 
Asset Items 



All Students 



• Student cash/ 
savings^ checking 

• Student home 
value 

• Student home debt 

• Student business/ 
farm value 

• Student real 
estate/ invesunent 
value 

• Student real 
estate/ investment 
debt 



Parent Tax* 
Items 



Dependent 
Students Only 



• Parents adjusted 
gross income 

• Parent U.S. 
taxes paid 

• Parent work 
income 

• Parent's spouse 
work income 

• Parent itemized 
deductions 

• Parent tax form 

• Parent number 
of exemptions 



Parent 
Untaxed Items 



Dependent 
Stiidents Only 



Parent child 
support nxeived 
Parent Social 
Security benefits 
Parent AFDC 
received 



Parent 
Asset Items 



Dependent 
Students Only 



• Parent cash/ 
savings/ checking 

• Parent home value 

• Parent home debt 

• Parent business/ 
farm value 

• Parent real estate/ 
investment value 

• Parent real estate/ 
investment debt 
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reported values for each application item in a group with the 'l)est'* values for each item 
in the group. 



For example, marginal error for Student Untaxed Income is calculated using best values 
for child support received, social security benefits, AFDC received, and other untaxed 
income. A new SAI and FC are computed using the 1)est" values for the grouped 
application items, and these are used to recompute the best awards and certifications. 
The difference between the award calculated with the substituted "best" values for ail 
items in the group and the award calculated using only reported values represents the 
marginal error associated with that sSSliD. of application items. 

All student marginal errors groupings were calculated using a $50 tolerance, and 
measured for their effect on Pell absolute award, Campus-Based need, and Stafford Loan 
overcertification. Exhibit HI-? presents the groups of student marginal errors tested, the 
percent of students in error, and the contribution to dollar error associated with each 
group of application items. Grouping the student marginal errors provided the following 
findings: 

• Student Tax Items as a group contribute most to error among the variable 
groupings reported by all students and across all awards: Pell (2.7 
percent), Campus-Based (3.5 percent), and Stafford (7.7 percent). 

• Parent Asset Items and Parent Tax Items are both high contributors to 
student error in dependent students across all programs. Parent Asset 
items as a group are by far the largest contributors to Pell dollar error (2.8 
percent of dollars in error). 



ERIC 



IIM8 

13;) 



ERIC 



GROUPED STUDENT MARGINAL ERROR 
Award Year 1988-89 




Dependency 
Status Items 



Student Tax 
Items 



Student Untaxed 
Income 
Items 



Student Asset 
Items 



Parent Tax 
Items 

(Dependents 
Only) 



Parent 
Untaxed Income 
Items 

(Dependents 
Only) 



Parent Asset 
Items 

(Dependents 
Only) 



Pell 

Campus-Based 
Stafford Loan 



Unweighted Avg, 



Pell 

Campus-Based 
Stafford Loan 



Unweighted Avg. 



Pell 

Campus-Based 
Stafford Loan 



Unweighted Avg. 



Pell 

Campus-Based 
Stafford Loan 



Unweighted Avg. 



Pell 

Campus-Based 
Stafford Loan 



Unweighted Avg 



Pell 

Campus-Based 
Stafford Loan 



Unweighted Avg 



Peli 

Campus-Based 
Stafford Loan 



Unweighted Avg 



0.9 
2.0 
1.3 
1.4 



2.7 
7.7 
3.5 
4.6 



1.7 
2.6 
1.6 
2.0 



1.2 
1.6 
0.8 
1.2 



5.8 
10.6 
3.0 
6.5 



3.7 
2.8 
4.4 
3.6 



8.3 
12.2 
8.0 
9.0 




0.8 
0.8 
1.1 
0.9 



0.5 
0.5 
0.4 



0.5 



0.4 
0.3 
0.4 
0.4 



1.6 
1.5 
1.1 
1.4 



0.7 
0.4 
1.2 
0.8 



2.8 
1.6 
1.9 



2.1 



665 
1,382 
1,590 
1,212 



437 
463 
812 
571 



453 
880 
733 
689 



495 
922 
1,248 
888 



355 
554 
835 
581 



238 
503 
633 



458 



440 
529 
543 
504 



1 All Student Marginal Error was measured for its effect on: (1) Pell Award, (2) Campus-Based need, and 
(3) Stafford Loan Overccrtification. III-19 
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• About one in every ten dependent Campus-Based students had an error 
with at least one Parent Tax or one Parent Asset Item. 

• Although Dependency Status Items do not account for large number of 
student errors, they do have a large effect on dollar error when they are 
missed. Campus-Based students had an average need error of $1,382 and 
Stafford loan recipients had an average overcertification error of $1,590. 



C. Institutional Marginal Error 

Institutional marginal error is reported separately for three categories: procedural error, 
calculation error, and distribution error. Procedural error refers to the error caused by 
an institution's failure to follow the established guidelines for granting awards (e.g., 
disbursing a Pell Grant without a signed Statement of Educational Purpose in the 
student's file, or disbursing more than one payment without a Financial Aid Transcript in 
the student's file). Calculation error refers to the error caused by an institution using 
inconect information to calaUate an award or by a mathematical error in performing the 
award calculation. Using an incorrect Pell award to compute Campus-Based need, or 
using the wrong cost of attendance in calculating a Stafford Loan certification are 
examples of calculation error. Distribution error refers to the error caused by an 
institution disbursing an incorrect amount with respect to the expected award or 
certification. Allowing a Stafford Loan to exceed the certification, or distributing 
Campus-Based aid in excess of need are examples of distribution error. 
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Institution marginal enor is calculated by substituting the institution reported value for a 
single award calculation component (e.g., Pell cost of attendance) with its "best" value. 
Best values for the calculation components of each award were obtained via award- 
specific questions in the Student Record Abstract. New awards and certifications are 
then calculated using the best value for, in this case. Pell cost of attendance. The 
difference between the award calculated with the substituted best value and the award 
calculated using only reported values represents the marginal error associated with the 
substituted calculation component (Pell cost of attendance). 



Procedural error and calculation error are measured for their effect on Pell absolute 
award, Campus-Based need, and Stafford overcertification. However, because 
distribution errors effect award distribution without effecting need or certification, 
institutional distribution errors were measured for their effect only upon Campus-Based 
distribution and Stafford Loan Overaward error. All of the institution marginal errors 
were calculated using a $50 tolerance on the error. Examination of the significant 
institution marginal errors across all programs led to the following findings: 



• Procedural error does not contribute as much to institutional error as does 
either Calculation error or Distribution error. 

• While the frequency of procedural errors is small, mean error per student 
with error is large, especially in the Campus-Based and Stafford Loan 
programs. This is due to the definition Procedural error: if a procedural 
error occurs then the entire award is considered in error. 

• While the Pell Grant and Stafford Loan are typically overawarded and 
overcertified as a result of institutional errors, the Campus-Based programs 
are typically underawarded. 
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• Institutional marginal error is dominated largely by calculation errors, 

which include the highest institutional marginal errors tested. Calculation 
error is dominated by errors in factoring the Pell award and all other aid. 

1. Procedural Error 

Procedural error occurs when an institution fails to follow the established guidelines for 
granting awards (e.g., disbursing a Pell Grant without a signed Statement of Educational 
Purpose in the student's file). Procedural errors are often referred to as categorical 
errors; if a student has a procedural error, he or she is ineligible for any financial aid, 
and the entire award (or the amount disbursed after the first payment in the case of 
Financial Aid Transcript error) is considered to be in error. For example, not having a 
signed Statement of Academic Progress in the smdent's file is considered a procedural 
error and the student is not eUgible to receive financial aid. 

Overall institutional error rates were found in 4.6 percent of all Pell students (Pell 
Award Error), in 20.9 percent of all Campus-Based students (Campus-Based need error), 
in 9.6 percent of all students receiving a Stafford Loan (Stafford Loan overcertification 
error). Examining procedural error further decomposes these error rates and attributes 
the overall error to individual procedural requirements. 

Exhibit III-8 presents the components of procedural error, the percent of students in 
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MARr.TNAT. TNSTTTlJ HONAL ERROR PROCEDURAL ERROR 
Pell Award 



Procediiral Error Type 




Bachelor's Degree 

Statement of Educational Purpose 

Selective Service Compliance 

Financial Aid Transcript 

Independent: no documentation 

Statement of Academic Progress 

Ability to Benefit 

Default/Repayment 

U.S. Citizenship 

Half Time Enrolhnent 



Grouped^ 



0.7 

0.5 

0.4 

0.4 

0.1 
* 

* 

* 

0 

0 



1.0 



0.6 
0.6 
0.6 
0.3 
0.1 
0.1 

* 

0 

0 



1.2 



1,278 
1,614 
1.923 

875 
1,886 
1,881 
1,896 

915 
0 
0 



1,690 



Campus-Based Need 


faReSBpleWt^ 






Procedural Error Type 


Statement of Educational Purpose 


0.4 


0.5 


5,111 


Financial Aid Transcript 


0.3 


* 


664 


Selective Service Compliance 


0.1 


0.1 


5,029 


Default/Repayment 


0.1 


0.1 


2,867 


Statement of Academic Progress 


0.1 


* 


2,445 


Bachelor's Degree (SEOG onljO 


0.1 


* 


511 


Ability to Benefit 




* 


7,627 


Independent: no documentation 


* 


* 


3.481 


U.S. Citizenship 


0 


0 


0 


Half Time Enrollment 


N/A 


N/A 


N/A 


Grouped^ 


0.7 


0.8 


4,635 



Stafford Loan Overcertirication 


.■•/Percent; of 
■■■"Recipient^ 


,j^JPerceiitv.;^;^; 


Mean Error I 
SPer Recipient 


Procecdural Error Type 


wlthEr^ 


^^^^^n^Errbr-'-^ 


■%lth-Eripor($); 


Half Time Enrollment 
Selective Service Compliance 
Statement of Educational Purpose 
Default/Repayment 
Financial Aid Transcript 
Independent: no documentation 
Statement of Academic Progress 
Ability to Benefit 
U.S. Citizenship 
Bachelor's Deeree 


1.3 

0.6 

0.5 

0.2 

0.2 

0.1 
* 

* 

0.0 
N/A 


1.2 

0.6 
0.6 
0.1 
0.2 

0.1 

* 

* 

0.0 
N/A 


2,443 
2,584 
3,140 
1,044 
3,397 
2,625 
3,210 
2,625 
0 

N/A 


Grouped ^ 


2.2 


2.1 


2,522 



* LESS THAN 0.05% 

1. Grouped errors do not include the effect of Bachelor's Degree or Financial Aid Transcript errors. 
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error, and the impact on dollar error for each of the thre'?' programs studied. Significant 
findings from the procedural error analysis include: 

• Procedural errors explain very little institutional error across program, but 
they do have high payment consequences (mean errors of $1,000 or more 
for recipients with error). 

• Bachelor's Degree error and Financial Aid Transcript error contribute 
significantly to Pell and Campus-Based procedural error. Statement of 
Educational Purpose and Selective Service compliance rsjik persistently 
high across programs. 

• Less than half time enrollment error dominates procedural error for 
Stafford Loan overcertification, but may be overstated since our data 
collection effort did not include confirmation from the institution on 
enrollment status. (Pell award did include an enrollment status check, and 
all error was explained by the Financial Aid Officer). 



2. Calculation Error 



Calculation error refers to the error caused by an institution using incorrect information 
to calculate an award. Calculation error includes errors in: (1) calculating the correct 
cost of attendance, and (2) factoring other financial aid. Calculation enor, like 
procedural error, is measured for the Pell award, Campus-Based need, and Stafford 
overcertification. However, errors in factoring other financial aid apply only to Campus- 
Based and Stafford Loan awards (the calculation of Pell award does not consider other 
financial aid). 



Exhibit III-9 presents the components of calculation error, the percent of students in 
error, and the contribution to dollar error made by each type of error. A review of 
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MAROTNAL IN STmiTfONAL ERROR 
rAiriTT.ATfON ERROR 

AWARD YEAR 1988-1989 



ERROKTYPE 


AWARD 
TYPE^ 


PERCENTOF 
RECIPIENTS 
WITH ERROR 


PERCENT OF 
DOLLARS IN 
ERROR 


MEAN ERROR 
PER RECIPIENT 
WITH ERROR 

VP/ 




PELL 


1.6 


0.2 


168 


COST OF 
ATTENDANCE 


CB 


2.2 


0.4 


820 




SL 


2.5 


1.0 


1,039 


FACTORING 
OTHER AID 


CB 
SL 


18.9 
4.4 


1.2 
1.2 


1,221 
759 



1. Calculation error was measured for its effect on: (1) Pell Award, (2) Campus-Based Need, and 
(3) Stafford Loan Overcertification 
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calculation error produced the following findings: 

• Cost of attendance error affects a greater percent of Stafford Loan 
recipients than Campus-Based or PeU recipients. Stafford Loan cost of 
attendance error also had high payment consequences (1.0 percent of 
dollars in error and an average $1,039 error for those students that had 
error). 

• Errors in Factoring Other Aid into Campus-Based need (includes Pell 
award, Stafford Loan certification and other financial assistance) is the 
greatest contributor to calculation error (18.9 percent of students in error 
out of 20.8 percent with a calculation error). 

• Errors in calculating the Campus-Based award present a costly component 
of institutional error: 5.6 percent of the dollars in error, and an average 
error of $1,191 per student with error. 



3. Distribution Error 

Distribution error refers to the error caused by an institution disbursing an incorrect 
amount with respect to the expected award or certification. Certifying a Stafford Loan 
which exceeds the maximum eligibility amount for that smdent and awardmg a Campus- 
Based package in excess of student need are examples of distribution error. Distribution 
errors include: 

• Disbursement Error Distributing a Pell disbursement inconsistent with the 
Student Aid Report; distributing Campus-Based aid inconsistent with the 
packaged Campus-Based award. 

• Initial Overaward Error Distributing Campus-Based aid which exceeds 
Campus-Based need; certifying a Stafford Loan which exceeds maximum 
loan eligibility. 
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Distribution errors effect the award distribution without effecting need certification; we 
therefore measure distribution errors for their effect upon Campus-Based distribution 
and Stafford Loan Overaward, instead of Campus-Based need and Stafford Loan 
certification. 



Exhibit ni-lO presents the two types of distribution error, the percent of students in 
error, and the impact on total dollars distributed for each of the programs affected. 
Significant findings from distribution error analysis include: 

• Campus-Based disbursement error contributes a significant amount to 
institutional error (9.7 percent of students in error). However, we did not 
find any cases of Pell Disbursement error. The fact that our data 
collection effort allowed a confirmation with the institution on Pell 
Disbursement errors but not Campus-Based Disbursement errors suggests 
that Campus-Based Disbursement error may be overstated. 

• Campus-Based overawards account for a greater number of institutional 
errors than do Stafford Loan overawards (2.7 and L8 percent of students 
with error, respectively). However, Stafford overawards have a greater 
impact on dollar error (1.3 percent of the dollars in error as compared to 
0.6 percent for Campus-Dased overawards). 



Of the three types of institution marginal errors studied, calculation error is by far the 
greatest contributor to error. While procedural and disbursement error are meaningful 
measures of institutional error, calculation error represents a persistent problem for 
institutions and, consequently, a source of high dollar error as well. 
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MARGTNAL mSTITimONAL ERROR 
DTSTRTBimON ERROR 



AWARD YEAR 1988-1989 



ERROKITPE 


TVPE 1 


PERCENT OF PERCENT OF MEANBRROR 
RECIPIENTS DOLLARS IN PER RECIPIENT 
WITH ERROR ERROR Wmi ERROR 

($) 


DISBURSEMENT 
ERROR 


PELL 
CB 


0.0 0.0 0.0 
9.7 1.5 679 


INITIAL 
OVERWARDS 


CB 

SL 


2.7 0.6 962 

1.8 1.3 1,911 



1. Distribution error was measured for its effect on: (1) Pell Award, (2) Campus Based Distribudon, and 
(3) Stafford Loan Overaward 
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IV. ERROR PROFILE ANALYSIS 

The marginal error analyses of the prior chapter examined the sources of error in 
student aid awards by identifying those data elements used in the award calculation 
that are most prone to error. This chapter presents the results of an error profile 
analysis to identify the characteristics of students and institutions that are associated 
with error. 

The error profile analysis has shown that income has the strongest relationship, among 
the variables tested, with student error. Independent students with income greater 
than $15,000 and dependent students whose parents have combined incomes of over 
$25,000 are predicted to have the highest error rate. The combination of awards that 
the student received was the variable found to have the strongest relationship to 
institutional error. These results are discussed in greater detail later in this chapter. 

A. Error Profile Analysis Methodology 

The error profile analysis conducted during IQCMP developed predictive models to 
identify characteristics of students and institutions that are associated with error. 
These models are not intended to be used by ED to target individual students and 
institutions with error. Rather, predictive models help to identify relationships 
between error and the character itics of students and institutions. An appreciation of 
these relationships furthers our understanding of the causes of error, leading to 
corrective actions that address those causes. 
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Separate analyses were conducted to determine the factors associated with student 
error and the factors associated with institutional error. As a first step in the 
analysis, two databases were created, one for the analysis of student error and one for 
the analysis of institutional error. 

For each student, we first identified whether any one of the three following types of 
errors had occurred: 

• Absolute Pell Program Error, 

• Absolute Campus-Based Need Error, and 

• Stafford Loan Overcertification Error. 

The student error analysis database included a variable indicating whether a student 
error of $50 or more had occurred for at least one of these three error measures. 
Similarly, the institutional error analysis database included a variable indicating 
whether or not an institutional error of $50 or more had occurred. 

The student and institutional error databases each included characteristics of both 
students and institutions that we believed might be related to error. For example, the 
student error analysis database included: 

• the information reported by the student on the financial aid application, 

• the type of aid received by the student, 

• the characteristics of the institution attended by the student (institution 
type and control), 
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• the procedures used by the institution to verify student data, and 

• questions from the student interview about the student's perception of: 

- the availability of the information needed to complete the 
application, 

the complexity of the appUcation, and 

the amount of help received while filling out the application fonn. 

The institutional error analysis database included characteristics of the institution (e.g., 
type, control, region, etc.) and information on institutional procedures for processing 
and checking the award (level of automation, procedures for verifying that the correct 
information was in the student's financial aid file, staffing in the financial aid office, 
etc.). 

We formed categories for all of the variables included in the database. For 
categorical variables (e.g., yes/no questions, institution type and control, etc.), the 
predefined categories were used, or were collapsed into fewer categories when the 
combination was applicable. For continuous variables (e.g., AGI, income, home 
equity, etc.), categories were formed through a ranking process. Each value of a 
variable was assigned a score equal to the percentile represented by that value (e.g., 
the median value for family income was replaced by a score of 50). These scores 
were all between 0 and 100. The scores were then partitioned into the following 
categories: 
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• 10 or less, 

• 11-25, 

• 26 - 50, 

• 51 . 75, 

• 76 - 90, or 

• 91 - 100. 

After all the variables were categorized using one of the above methods, contingency 
tables were formed that compared the categories of each variable to the presence of 
error (using a $50 tolerance). Chi-squared analysis, a standard statistical technique 
that measures the association between two categorical variables, was conducted on 
each variable. Exhibits D-1 and D-2 in Appendix D present examples of the chi- 
squared analysis for an institutional and a student variable. The results of this 
analysis are discussed in the following sections. 

B. Variables Associated With Error 

1. Characteristics Associated With Student Error 

Each variable tested for association with student error was tested on the relevant 
population of independent, dependent, or all students. Parent data (e.g., parent's 
Adjusted Gross Income and the number in the parent's household), which are 
reported only by dependent students, were tested on a subset of the database 
containing only dependent students. Similarly, variables which are reported only by 
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independent students (e.g., student's Household Size) were tested using a subs< of the 
database containing only independent students. Variables reported by both 
independent and dependent students (e.g., student's Adjusted Gross Income) were 
tested using the full database of all students. Exhibit B-1 (Appendix B) presents all 
the variables that were tested for association with student error and the results of the 
test. 

We found that most of the variables related to student error are associated with 
income (e.g., total income, family income, net family assets, etc.). A few variables 
describe general characteristics of the student (e.g., year in college, received help in 
filing Financial Aid Application, etc.). 

2. Characteristics Associated With Institutional Error 

Exhibit B-5 (Appendix B) lists the variables tested for association with institutional 
error. Only a few of the variables (e.g., institution type, institution control, etc.) were 
found to have a significant relationship with institutional error. Further, two of the 
variables that were related to error (i.e., policy regarding the collection of tax forms 
and the methods that were used to inform students about the importance of accurate 
data and the consequences of misreporting) seem to bear little relationship to 
institutional error, unless they are surrogates for the dedication of the institution to 
quality control. Because this is a tenuous interpretation at best, these variables were 
dropped from further analysis. In addition, two measures of size that showed a 
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Statistically significant relationship with error, awards per full time equivalent staff and 
number of awards, did not follow a consistently increasing or decreasing trend, so the 
relationships were deemed spurious. 

C Error Prediction Model 

Although the contingency table analysis discussed above is useful in examining 
characteristics associated with enor, it cannot account for the effect of relationships 
between those characteristics. For example, all of the parent characteristics associated 
with student error are fJso related to income. While parent's AFDC received is 
related to student error, it is possible that the driving force behind this relationship is 
a low parent income. In order to determine whether there is a true relationship 
between the amount of the parent's AFDC and error, the value of parent's income 
must be controlled in the analysis. ControlUng the value of one variable while testing 
another is not possible in a contingency table analysis; multivariate techniques must be 
used. 



One problem to be overcome in the multivariate analysis is that, when many variables 
are included in the model and each variable has several categories, the model 
includes so many parameters that it is Ukely to pick up spurious relationships. To 
overcome this problem, we employed an approach developed by the IRS as a part of 
its methodology to select tax returns to be audited. This approach involves a variable 
transformation based on likelihood ratios, as described below. 
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As the first step in the multivariate analysis of characteristics related to error, the 
results of the contingency analysis were used to assign a value to each variable based 
on the proportion of students with error for the relevant category. For example, 
suppose a student had AGI of $0 or less. This student would be assigned a value of 
31.97 for AGI because, referring to Exhibit D-1, 31.97 percent of students with AGI 
of $0 or less had a student enor. Similarly, a student with AGI between $1 and $600 
would be assigned a value of 44.22 for AGI because, as can be seen from Exhibit D- 
1, 44.22 percent of the students with AGI between $1 and $600 had an error. The 
effect of this variable transformation is to create a single continuous variable for AGI 
to replace the six categorical AGI groupings. 

After performing the variable transformation, we reduced the number of variables to 
be included in the model using a stepwise regression analysis. Stepwise regression is 
a data exploration technique where variables are entered in the equation one at a 
time until a prespecified maximum is reached. At each step, the variable that causes 
the largest increase in predictive ability (as measured by the model's statistic) is 
added to the model, then each variable in the model is compared to each variable 
not in the model to determine whether replacing a variable will yield a higher R^. 
The process continues until the "best" one variable model is found, the "best" two 
variable model is found, and so on. The results of this process were examined for 
reasonability and used as a starting point for the final step in the model building 
process. 
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After using stepwise regression to reduce the number of variables to be included in 
the model, we returned to a categorical analysis of the variables remaining. We 
employed a multivariate analysis technique known as Analysis of Variance (ANOVA) 
to estimate the relationship between student and institutional characteristics and error. 
During the this part of the analysis some variable categories were reHned based on 
the results of the model (i.e., two categories of a variable that had a similar 
coefficient were combined). The sections below describe the results and implications 
of the error prediction models developed during this process. 

1. Error Prediction Model for Student Error 

Many of the variables that were found to be related to student error during the 
contingency table analysis were not found to be significant predictors of error in the 
muhivariate model (e.g., AFDC was shown not to relate to error when income was 
included in the model). Exhibit IV-1 presents the variables that are in the student 
error prediction model to for independent students and the relative contribution of 
each level of each variable. Exhibit IV-2 presents the same information for 
dependent student error prediction model. 

Exliibits IV-1 and IV-2 show that income has a strong relationship with student error. 
Independent students with income over $15,000 are pr dieted to have student error 
18.7 percent more often than independent students with income under $7,500, and 
dependent students whose parents have income over 525,000 are predicted to have 
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EXHIBIT IV-1 

— 1 

Error Prediction Equation for Independent Student Characteristics 

Includes Relative Contributioo to Error Probability (in percent) 

Intercept (-15.8) 

Total Student Income 

$0 - $7^ (0.0) 
$7^ - $15,000 (8i) 

• Over $15,000 (18.7) 

Tax Form Filed 

Student filed a tax form (7.9) 

• Student did not file a tax form (0.0) 

Used Estimated Income Tax Data 

• Used estimated tax data when filling in application (10.2) 

• Did not use estimated tax data when filling in application (0.0) 

Student's Untaxed income 

• Student has untaxed income (3.7) 

• Student does not have untaxed income (0.0) 

Indicator for Pell Award 

• Student received Pell Grant (15.7) 

• Student did not receive Pell Grant (0.0) 

Indicator for Campus-Based Award 

• Student received Campus-Based Award (8.9) 

• Student did not receive a Campus-Based Award (0.0) 

Indicator for Stafford Loan 

• Student was certified for Stafford Loan (0.2) 

• Student was not certified for a Stafford Loan (0.0) 
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EXHIBIT 1V.2 

I 

Error Prediction Equation for Dependent Student Characteristics 

Includes Relative Cootributioo to Error Probability (in percent) 



Intercept (-12.2) 

Total Parent Income 

$0* $15,000 (0.0) 
$15,000 - $25,000 (28.9) 
Over $25,000 (30^) 

Tax Form Filed 

• Parent filed a tax form (9.9) 

• Parent did not file a tax form (0.0) 

Used Estimated Income Tax Data 

• Student or parent used estimated tax data when filling in application (12.6) 

• Neither student nor parent used estimated tax data when filUng in application 
(0.0) 

Net Value of Parent's Real Estate and Other Investments * 
$0 - $500 (0.0) 
Over $500 (OS) 

Indicator for Pell Award 

• Student received Pcll Grant (15.7) 

• Student did not receive Pell Grant (0.0) 

Indicator for Campus-Based Award 

• Student received Campus-Based Award (10.6) 

• Student did not receive a Campus^Based Award (0.0) 

Indicator for Stafford Loan 

• Student was certified for Stafford Loan (5.6) 

• Student was not certified for a Stafford Loan (0.0) 



Includes the vilue (after debu) of investment real estate, cash, savings and checking accounu, and other 
investmenu. Does not include investments in homes, businesses, or farms. ^ 
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Student error 30.8 percent more often than dependent students whose parents have 
less than $15,000 in income. The fact that the model's income coefficients are greater 
than the coefficients for other variables implies that income is the most important 
factor associated with error. 

The strong effect of income helps explains why many of the other factors initially 
found to be significantly related to error in the contingency table analysis were not 
found to be significant in the multivariate model. Most of these other variables were 
related to income, so that when considered by themselves they partially captured the 
relationship between income and error. After controlling for income, however, the 
other variables did not add significantly to the predictive ability of the model. 

There are at least two reasons why high income is correlated with error. First , 
students and parents with high income generally have more complex financial 
situations and, therefore, generally have more oppormnities to make errors in the 
financial aid application. Low income students and parents are less likely to have 
significant assets, medical/dental expenses, elementary and secondary tuition expenses, 
etc. Thus, their application process is simpler and less error prone. Secondly , many 
low income students and parents have financial need sufficiently high to justify a full 
award even after errors in their financial aid application are corrected. Thus, errors 
committed by low income students and parents are less likely to affect the financial 
aid award. 



ERIC 



IV-U 

151; 



IQCMP Findin ffs and Correctivg Actions 



Many of the other variables found to be significant predictors of error in Exhibits IV- 
1 and IV-2 are, like income, related to the complexity of the student's financial 
situation and the financial need of the student. Higher error rates were found for 
students or parents who filed a tax. return, used estimated tax data rather than actual 
tax data, had untaxed income, or had significant real estate or investment assets. 

We also found that independent students who received help in completing their 
financial aid application were more likely to have an error than those who completed 
the application on their owr or those who had someone else (presumably their 
parents or financial aid counselors) complete the application for them. However, the 
effect of this variable on error was not nearly as large as the effect of income and tax 
return filing variables. 

Finally, both the independent and dependent student models take into account the 
financial aid received by the student. Students receiving Pell and/or Campus-Based 
aid were predicted to be particularly more enor prone than those receiving only 
Stafford loans, after controlling for differences in income, tax return filing, etc. This 
result probably occurs because, as shown in Chapter II, Stafford Loan overcertification 
error occurs less frequently than Pell error and Campus-Based need error. 

Exhibit IV-3 summarizes the accuracy of the dependent and independent student 
models combined. To prepare this exhibit, we first combined the independent and 
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dependent student models and used the combined models to predict the likelihood 
that each individual student had an error. All students (both dependent and 
independent combined) were then sorted from highest to lowest predicted error 
likelihood. We identified the group of students with the highest predicted error 
likelihood, the group of students with the two highest predicted error likelihoods, the 
group with the three highest predicted error likelihoods, and so on. For each of 
these student groups, we placed a point on Exhibit IV-3 based on the cumulative 
percent of all students represented by the group (the horizontal axis of Exhibit IV-3) 
and the percent of students in the group that actually had student error (the vertical 
axis of the exhibit). This line is the middle curve on Exhibit IV-3. 

For example, the point on the middle curve in Exhibit IV-3 located at about 30 
percent on the horizontal axis and about 60 percent on the vertical axis implies that 
the model can be used to select a group of 30 percent of the students so that 60 
percent of the cases selected have a student error. Similarly, the point on the middle 
curve at about 50 percent on both the horizontal and vertical axes implies that the 
model can be used to select a group of 50 percent of the students for which 50 
percent have a student error, As the size of the group selected by the model 
increases, the percentage of the group in error decreases until it reaches to just under 
30 percent, the overall student error rate. 
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The graph indicates that if a small percentage of the students are chosen using this 
model, the group can be chosen so that all students in the group have error. While 
this result is true for the sample of students examined during IQCMP, it is probably a 
by-product of the model fitting procedure used and should not be expected to hold 
for the student population in general. 

The top line in Exhibit IV-3 represents an ideal model. This ideal model would 
select up to nearly 30 percent of the students in such a way that each selected 
student has a student error (represented by the horizontal line at 100 percent). If 
more than 30 percent of all students are selected, the percentage of the group 
selected which has a student error will drop in a linear fashion because, after 
selecting the 30 percent with error, the model will start to select students that did not 
have an error. If all of the students were selected, about 30 percent of the group will 
have student error. 

The horizontal line at just under 30 percent represents the percentage of students in 
error that would be selected by a random selection model. This line represent a 
model that has no predictivf; abiUty. 

2. Error Prediction Model for Institutional Errors 

As was expected from the results of the contingency table analysis, few variables were 
related to institutional error. The final error prediction model for institutional enors 
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includes only the variables shown in Exhibit IV-4. 

As with the analysis of student error, Exhibit IV4 shows that institutional error is in 
large part related to the financial aid received by the student. For example, students 
receiving Campus-Based aid are predicted to be nearly 19 percent more likely to have 
an institutional error than nonrecipients. This is to be expected because institutional 
error for Campus-Based need was much higher than either Stafford Loan 
overcertification error or Pell award error. The relative order of the effects of awards 
received in the model (Campus-Based higher than Stafford Loan, and Stafford Loan 
higher than Pell) is identical to the order of institutional error rates among the three 
programs. 

Exhibit IV-4 also shows that students attending proprietary institutions or attending 
institutions that do not recheck institutional records for the required documentation 
are more likely to have institutional error. Students attending 4-year baccalaureate or 
graduate degree-granting programs were only slightly less likely to have an 
institutional error than those attending other institutions. 

Exhibit IV-5 presents a graph of the performance of the selection model for 
institutional error. As with Exhibit IV-5, the top line on the exhibit represents a 
perfect selection model and the bottom line on the exhibit represents a model with 
no predictive ability. The curve between these two lines is the result of selecting 
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EXHIBIT IV.4 



Error Prediction Model for Institutional Characteristics 

Includes Relative Coatribution to Error Probability (in percent) 



Intercept (-2.3) 

Institution Control 

• Public or Private (0.0) 

• Proprietary (8.0) 

Institution Type 

• 0-4 Year Program (2.9) 

• 4 Year Baccalaureate and Graduate Programs (0.0) 

Method used to re-check files for documentation 

• Automated system (0.0) 

• All manual systems (2.7) 

• Do not re-check (8.6) 

Indicator for Pell Award 

• Student received Pell Grant (3.2) 

• Student did not receive Pell Grant (0.0) 

Indicator for Campus-Based Award 

• Student received Campus-Based Award (18.7) 

• Student did not receive a Campus-Based Award (0.0) 

Indicator for Stafford Loan 

• Student was certified for Stafford Loan (7.2) 

• Student was not certified for a Stafford Loan (0.0) 
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students based on the institutional error prediction model. As shown on the graph, 
the model significantly improves on the random error model. For example, using this 
model a group of 10 percent of all students could be selected so that approximately 
40 percent of the students had an institutional error. The predictive ability of the 
model drops to about 30 percent when 30 percent of the students are selected, and 
reduces approximately linearly beyond that point. 
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Department of Education regulations for Integrated Verification require institutions to 
verify certain applicant-reported data for a sample of Tide IV applicants. In addition, 
many institutions supplement their Integrated Verification activities with an Institutional 
Verification program. This section examines the effect of Integrated Verification and 
Institutional Verification on student error. 

A. Integrated Verification Practices 

Under Integrated Verification, the central processor, Multiple Data Entry processors, 
and certain Need Analysis Servksrs use edits developed by ED to select Title IV 
applicants for verification. If more than 30 percent of an institution's Title IV applicants 
are selected for Integrated Verification, the institution may choose to verify only a 
subsample of those selected. (The subsample must include at least 30 percent of the 
institution's Title IV applicants.) Exhibit V-1 summarizes the practices of institutions 
with regard to the 30 percent rule and the number of Title IV recipients attending those 
institutions. 

Exhibit V-1 shows that most institutions (over 84 percent) verify all ED-selected 
applicants. Furthermore, because those institutions that do not verify all ED-selected 
applicants often do not have many more than 30 percent of their appHcants selected, we 
found that only 1.2 percent of the Pell recipients selected for Integrated Verification 
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Exhibit V-l 

Percentage of ED»Selected Students Verified 



Percent of 

What percentage of ED-selected students Percent of Students 
do you verify? Institutions Affected 



a. Only 30 percent of all applicants, 

as required by law 10.0 12.3 

b. More than 30 percent of all applicants, 

but not all ED-selected applicants 5.8 4.1 



c. All ED-selected applicants 84.2 83.7 



100.0 100.0 



were not verified because of the 30 percent rule. If less than 30 percent of an 
institution's applicants are selected by the processor, the institution is required to select 
additional applicants for verification from among those that were not subjected to the 
edits (provided the institution has applicants that were not subjected to the edits). We 
did not collect data that would enable us to identify how institutions are affected by this 
requirement. 



When an applicant is selected for Integrated Verification, the institution must collect 
documentation to verify the following items on the financial aid application: 
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• Adjusted gross income 

• U.S. income taxes paid 

• Untaxed income and benefits 

• Household size 

• Number in college 

The Department's Verification Guide provides guidelines for acceptable documentation. 
The allowable documentation varies depending on the student's situation. For example, 
if the student has filed a tax return with the IRS, a copy of the student's tax return and 
Form W-2 should be used to confirm AGI, taxes paid, and certain untaxed income items. 
If the smdent will not file a tax return, the student should submit a signed statement 
stating that a return will not be filed and listing any income received. For household 
size and number in college, a signed statement is usually used as documentation. 
However, the Verification Guide states that additional documentation should be 
collected if the institution has reason to doubt the statement. 

Our data collectors reviewed institutional records to identify the documentation used to 
verify applicant-reported data. Exhibit V-2 summarizes the types of documentation that 
were found in financial aid files for Pell recipients selected by the edits for Integrated 
Verification. (We do not have information sufficient to identify non-Pell recipients 
selected by the edits.) The exhibit categorizes the documentation used for Integrated 
Verification into four groups: 
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nOriJMENTATTON SOURCES FOR 
TNTEfiRATKn VFRTFTCATION 





DocmHiiiatkw 

(PtfCWt) 



Doeamentatioii 

ijfttctat} i 



Statement 
Signed By 

(Pwant) 



No 

DocomcntatkMi 



total 



ADJUSTED 
GROSS INCOME 



Student 
Parent 



3.1 
5.3 



76.4 
87.6 



4.2 
2.4 



16.3 
4.7 



100.0 
100.0 



U5. TAXES 



Student 
Parent 



2.5 
5.6 



75.1 
87.2 



2.7 
1.7 



19.7 
5.5 



100.0 
100.0 



WORKINCOME 



Student 
Parent 



1.3 
2.5 



58.8 
45.6 



8.2 
1.8 



31.7 
50.1 



100.0 
100.0 



spousrs 

WORKINCOME 



Student 
Parent 



0.0 
2.3 



52.4 
60.1 



1.1 
0.9 



46.5 
36.7 



100.0 
100.0 



SOCIAL SECURTTYl 
BENEFITS 



Student 
Parent 



0.3 
1.5 



0.0 
9.6 



10.3 
9.2 



89.4 
79.7 



100.0 
100.0 



CHILDSUPPORt 
RECEIVED 



Student 
Parent 



HOUSEHOLD SIZE 



Student 
Parent 



0.6 
0.0 



0.0 
0.0 



12.9 
15.1 



4.3 
0.3 



2.9 
7.0 



64.8 
61.4 



NUMBER IN 
COLLEGE 



Student 
Parent 



15.8 
10.8 



0.0 
0.0 



26.7 
36.6 



86.5 
84.9 



28.0 
31.3 



57.5 
52.6 



100.0 
100.0 



100.0 
100.0 
100.0 
100.0 



1. Certified tax return, statement from employer, institutional records, statement from social service agency. 

2. Applicant-provided copy of tax return, W-2 Form, state tax form. 

3. Notarized statement, signed statement from parent/student, signed Verification Worksheet. 

4. Data collector could not find documentation in the file. Item may not have been report&a. 
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• Documentation Supplied by External Source 

- For income items (AGI, taxes paid, work income and untaxed 
income), institutions rarely collected documentation from external 
sources other than the student or parent (e.g., certified tax remms, 
signed statement from employer or social service agency). When 
such documentation was collected, the documentation was almost 
always a certified copy of the tax return supplied by the IRS. Other 
types of documentation collected included statements from the IRS 
that a tax return was not filed; statements by employers regarding 
work income; statements from the Social Security Administration on 
social security received by the applicant; and statements from social 
service agencies for child support received. 

~ For household size and number in college, institutions could 
sometimes confinn applicant reported data based on their own 
records (e.g., a number in college of one because the student 
attended the institution or a number in college of two because the 
student and a sibling attended the institution). Only in very rare 
cases did institutions contact other institutions to confirm number in 
college. 

• Documentation Supplied by Applicant 

Copies of Federal tax returns were frequently supplied by the 
applicant to document AGI, taxes paid, and work income. In some 
cases, the tax remm was also used to document social security and 
household size (presumably based on the number of tax 
exemptions). 

• Statement Signed by Applicant 

- Institutions frequently documented household size and number in 
college based on a signed statement from the applicant. In addition, 
institutions often document child support and social security based 
on a signed statement. Finally, when the applicant reported zero for 
income-related items (AGI, taxes paid, work income and untaxed 
income), institutions would collect a signed statement to suppo;rt 
this. 

• No Documentation 

Our data collectors sometimes could not find documentation in the 
student aid file to support applicant data. Most often, this occurred 
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when the applicant did not report the item (e.g., did not file a tax 
return or did not have taxed or untaxed income) or because the 
applicant reported mininin tti household size or number in college 
(e.g., household size of one for a single applicaat or two for a 
married applicant, number in college of one, etc.). 

Thus, there are two primaiy sources of documentation used for Integrated Verification: 
(1) income-related items are most commonly documented by a copy of a tax return 
supplied by the applicant, and (2) household size and number in college are most 
commonly documented by a signed statement. Documentation was often not found or 
not collected for income items that were reported as zero (e.g., no reported untaxed 
income) and for minimum values for household size and number in college (e.g., when 
an independent single student reports a household size and number in college of one). 

B. Institutional Verification Practices 

Many institutions have an Institutional Verification program under which they verify 
more than the minimum required number of students and/or more than the minimum 
required data items on the financial aid application. We asked financial aid 
administrators whether their institution selected additional students for institutional 
verification, and found that nearly three-quarters did. Exhibit V-3 provides the percent 
of institutions in the sample that verify appUcants not selected by ED, and the percent of 
sampled students affected by this policy. 



The percenlage of institutions selecting additional students (72.8 percent) is less than the 
percentage of students subject to these policies (84.1 percent), because institutions with a 
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Frequency of Institutional Verification 



Do you verify any applicants not selected 
for integrated verification by ED? 



a. Yes 



Percent of 
All 

Institutions 



72.8 



Percent of 

Students 

Affected 



84.1 




b. No 



27.2 



15.9 



100.0 



100.0 



J 



large number of recipients more frequently select additional students than institutions 
with fewer recipients. 

Exhibit V4 summarizes the methods used by institutions to select applicants for 
Institutional Verification. Nearly 43 percent of the institutions that verify additional 
students stated that they verify all applicants. These institutions tend to be institutions 
with fewer recipients (the 42.9 percent of institutions represent only 36 percent of the 
students). One-third of the institutions said they verify apphcants when conflicting data 
is submitted. Nearly 12 percent verify applicant data in error-prone circumstances. Only 
3.5 percent use selection criteria, and less than 1.0 percent select apphcants at random. 
Those institutions included in the "other" category frequently used a combination of the 
other criteria or targeted specific groups of students (e.g., all Pell recipients). 
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Exhibit V-4 



Selection of Applicants fo r Institutional Verification 



How do you determine which applicants 
to verify? 



Percent of 

Institutions 

Performing 

Additional 

Verification 



Percent of 
Students 
Subject to 
Additional 
Verificadcn 



a. 


Verify all applicants 


42.9 


36.0 


b. 


Conflicting documentation 


333 


31.8 


c. 


Error-prone circumstances 


11.7 


12.7 


d. 


Use selection criteria 


3.5 


8.4 


e. 


Randomly sample applicants 


0.7 


0.6 


f. 


Other 


7.9 


10.5 



100.0 



100.0 
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Exhibit V-5 summarizes the procedures used by institutions to verify applicant-reported 
data. Most institutions (nearly 80 percent) reported they followed the procedures 
required under Integrated Verification. The institutions that perform additional steps 
tend to have more recipients than other institutions since they represent 29.1 percent of 
the recipients and only 15.6 percent of the institutions. Similarly, the 5.4 percent of 
institutions that perform fewer steps tend to have fewer recipients since they represent 
only 2.3 percent of the recipients. 
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Institutional Verification Procedures 



Do you follow the procedures for 
Integrated Verification when you 
perform Institutional Verification? 



a. Perform same steps 

b. Perform additional steps 

c. Perform fewer steps 



Percent of 

Institutions 

Performing 

Additional 

Institutional 

Verification 



79.0 
15.6 
5.4 



Percent of 

Students 

Subject to 

AdrJitional 

institutional 

Verificadcn 



68.6 
29.1 
23 



100.0 



100.0 



Exhibit V-6 summarizes the data items they verified as a part of Institutional 
Verification. About two-thirds of the institutions reported that they verify only the data 
items used in Integrated Verification. Nearly 15 percent of the institutions verify data 
items when conflicting information is found or on a case-by-case basis. Another 9 
percent verify only those items on the tax remm. A few institutions verify all data items. 
Those responding "other" typically used a combination of the above. 
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Exhibit V-6 



Data Items Verifieii 



jgrated Verification 



How do you identify the data items to be 
verified for institution-selected students? 



a. Use ED verification items 

b. Conflicting data/case-by-case 

c. Tax return items 

d. Verify all data items 



e. 



Other 



Percent of Percent of 

Institutions Students 

Performing Subject to 

Additional Additional 

Verification Verification 



66.0 
14.9 
9.0 
2.6 
7.5 

100.0 



59.7 
13.0 
13.0 
4.2 
5.7 

100.0 



C Effect of Verification on Student Error 

During visits to sampled institutions, our data collectors recorded the information 
initially submitted to the institution by each student and the information ultimately used 
by the institution to determine the student's financial aid award. Differences between 
these two sets of data are primarily due to verification, although in some instances 
differences are due to corrections initiated voluntarily by students. We used this data to 
compute the effect of Integrated and Institutional Verification on student error. 
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Exhibit V-7 summarizes the error attributable to two sources: (1) incorrect student data 
in the information initially submitted to institutions, and (2) incorrect student data 
ultimately used by the institution to determine financial aid awards. The difference 
between these two error measures is primarily due to the effects of verification. Exhibit 
V-7 provides error measures for three groups of students: (1) students selected by the 
processor for verification, (2) students selected by the institution for verification, and (3) 
students not selected for verification. Students that were selected by the processor for 
Integrated Verification, but were not verified due to the 30 percent rule, are included in 
the "Not Verified" group. Exhibit V-7 only includes students who received a Pell award 
because we did not collect data sufficient to identify the verification status of non-Pell 
recipients. The exhibit does, however, evaluate the effects of verification on Campus- 
Based and Stafford Loan awards to students who also received a Pell award. 

For Pell awards, Exhibit V-7 shows that the error for students selected by the processor 
for verification (and verified by the institution) was reduced from an initial level of 33.4 
percent of students to a final level of 26.7 percent, a reduction of 6.7 percent. In terms 
of award amount, the percent of dollars in enor was reduced firom 12.7 percent to 9.4 
percent. 

Error in Pell recipients selected for verification by the institution was reduced from 30.9 
percent of the students to 20.7 percent of the students, and from 10.7 percent of the 
award dollars to 5.9 percent of the awards dollars. While these reductions in Pell error 
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Exhibit V-7 

EFFECT OF VERTFTCATTON ON STUDENT FRROR TN 

THE TITLE IV PROGRAMS 
Award Year 1988-89 

1 

PELL RECIPIENTS ONLY 



* ^ U ■'■ .: 




SRROrI CHANGE 




PROCESSOItSlUCim 

/ ^yv 



551 


33.4 


26.7 


-6.7 


12.7 


9.4 


-3.3 


260 


30.9 


20.7 


-10.2 


10.7 


5.9 


-4.8 


858 


25.8 


24.0 


-1.8 


7.8 


6.5 


-1.3 



mm mm 

INSIITUTIOHSBLECrED 
KOTVERIEIED' 




274 
144 
492 



47.9 33.9 -14.0 
47.8 39.8 -8.0 
39.7 36.1 -3.6 



Srr>JFFOR0L0AK 
OVCRCERTMCAttON 

PROCESsbit SELECTED 

INSTmmON SELECTED 

NOTVERFtED 



359 
155 
549 



14.8 
14.4 
10.5 



10.3 
12.9 
9.3 



-4.5 
-1.5 
-1.2 



ARITHMETIC AVERAGE OF 
ABOVE 

PROCESSOR SELECTED 

INSTITUTION ciELECTED 

NOT VERIFIED 



1 



N/A 
N/A 
N/A 



32.0 
31.0 
25.3 



23.6 
24.5 
23.1 



-8.4 
■6.5 
-2.2 



5.0 
5.7 
5.5 



4.0 
3.6 
3.8 



7.2 
6.7 
5.7 



2.7 
3.9 
4.6 



2.6. 

4.1 

3.4 



4.9 
4.6 
4.8 



-2.3 
-1.8 
-0.9 



-1.4 
0.5 
-0.4 



-2.3 
-2.1 
-0.9 
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1 . Recipients who did not receive a Pell are excluded. Total sample 1,669 recipients. 

2. Error in applicant's initial submission to the institution. 

3. Error in Tinal applicant data used by the institution to calculate award. 
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for institution-selected cases exceeded those obtained for ED-selected cases, the 
reductions in Campus-Based and Stafford Loan error for Pell recipients selected for 
Institutional Verification were less than those for ED-selected cases. Averaged across 
the three programs, processor-selected cases had a greater reduction in error than 
institution-selected cases. 

Exhibit V-8 presents a table similar to Exhibit V 7 which considers only errors in those 
student items that must be verified through Integrated Verification. While lower in 
magnitude, the values of initial error, final error, and error removed in Exhibit V-8 are 
similar to those in Exhibit V-7. 

Exhibit V-9 compares Pell error removed through verification for household size and 
number in college (which are most commonly documented by a signed statement) to that 
of AGI and V S. taxes paid (which are most commonly documented based on an 
applicant-provided copy of a tax remm). The exhibit tabulates the number of times that 
the value initially submitted by the applicant would cause a Pell award error and the 
number of times that the final verified value used by the institution would cause a Pell 
award error. Unweighted sample totals are used because the sample size is too small to 
make statistically reUable generalizations to the population. The results strongly suggest 
that verification based on signed statements does little to reduce error, since only five of 
the 103 initial errors in household size and number in college are corrected through 
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EFFECT OF VR RTFTCATTON ON STUDENT ERROR 

THE TITLE TV PROGRAMS 

Award Year 1988-89 
ERROR FROM VERIHC ATION ITEMS ONLY * 
PELL RECIPIENTS ONLY ^ 



Exhibit V-8 




NdirVERIFIED 



CAMPtIS B ASl^l^OLUTE 



PROCESSOir 
NOTVE 




271 
146 
495 



28.4 
33.0 
27.1 



20.0 
22.5 
24.6 



-4.1 
-4.5 
-2.5 



4.1 
4.5 
4.4 



2.4 
3.9 
3.5 



-1.7 
-0.6 
-0.9 
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STAFFORD LOAN 
OVERCERTIFICATION 

■|::;:;:|:|;-:::;.':::;:::::|:v^ 

PROCESSOR SELECTED 360 

INSITIWONSELECTED 158 

NOTVERIFIED 1 560 



ARITHMETICIVERAGEOF 
APOVE 

PROCESSOR SELECTED 

iNsrmjridNSEL^ 

■■■^not'verified-/^"^^^ 



18.7 
12.4 
12.7 



14.1 
8.0 
12.2 



-4.6 
-4.4 
-0.4 



4.8 
4.7 
3.6 



3.3 
4.0 
3.7 



-1.5 
-0.7 
0.1 



N/A 
N/A 
N/A 



24.1 17.9 -6.2 

21.2 14.8 -6.4 
19.7 18.4 -1.3 



5.6 
4.9 
4.5 



4.0 
4.2 
4.1 



-1.6 
-0.7 
-0.4 



1. Includes errors due to AGI, work income, taxes paid, liousehold size, number in college, social security benefits, child support 
received, and other untaxed income. 

2. Recipients who did not receive a Pell are excluded. Total sample 1,669 recipients. 

3. Error in applicant's initial submission U) the institution. 

4. Etfor in flnal apolicant data used by the institution to calculate award. 
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Exhibit V-9 



Pell Student Error Removed Througli Verification 
Processor-Selected Recipients Only 
(541 SaiLjjied Recipients) 



Number of Recipients With Error 



Variable 



Parent Household Size 
Student Household Size 
Parent No. in College 
Student No. in College 



Total 



Initial 
Error 



51 
12 
32 
8 

103 



Final 
Error 



50 
12 
29 
7 



Change 

-1 
-0 
-3 
-1 



98 



Parent Adj. Gross Inc. 20 

Student Adj. Gross Inc. 13 

Parent U.S. Taxes 26 

Student U.S. Taxes 12 



5 
9 
9 
8 



-15 
A 
•17 
-4 



Total 



71 



31 



-40 
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verification. (Signed statements do protect the institution from liabilities, because the 
school followed the regulatory process, even though the process appears to have a 
minimal effect on correcting error.) In contrast, 40 of the 71 initial enors in AGI and 
U.S. taxes paid are corrected through verification, indicating that verification based on 
copies of tax returns is an effective tool for reducing student error. 

D. Targeting of Verification 

As shown previously in Exhibit V-7, the initial error among students selected by 
processors for Integrated Verification did not differ substantially fi-om that of students 
not selected for verification. We therefore examined whether verification could be 
better targeted towards students with errors in the data elements verified through 
Integrated Verification. 

We created a database containing the values initially reported to institutions by students 
and the error attributable to the items verified during Integrated Verification. We then 
tested regression models to predict whether or not an error occurred based on the 
applicant-reported data. We found that two variables - dependency status and AGI 
were significant predictors of error. While multivariate models including additional 
variables were statistically significant, they produced only minimal improvements in 
predictive ability. Exhibit V-10 summarizes the results of our analysis. 
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Exhibit V-10 shows that dependent students are nearly three times as likely as 
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VERIFICATION COULD BE BETTER TARGETED 

ICMAEPS STUDEm^SJmiiJESBQR 



A. Dependent Students 




15,001 to 20»0QO 
20,001 10 ^,00O^ 
Over3S,000' 



20.4 
U.5 
13.6 
14.2 
30.7 
9.9 



;::A?:::-^:::::>::a::V^ 



10.4 
16.4 
30.8 
44.4 
52.7 
66.6 



419 
464 
320 
380 
446 
324 




34.8 



38.0 
49.1 
36.2 
26.7 



Alt Dependent 



100.0 



37.1 



399 



38.6 



B, Independent Students 





Percent of 
Students 


Percent With 
PellError^ 


Average Error Per 
Recipient Witit 
Error • 


Percent Selected 
By Processor 


Less Than iib^O^:;^ 


43.2 


4.7 


856 


24.5 


5,000 to 10,000 


26.7 


7.9 


437 


23.2 


. io,bottoi^o^ 


11.9 


22.3 


636 


28.4 




18.2 


34.4 


285 


54.0 


Alllndependent 


100.0 


12.8 


470 


30.2 



I Pell error attributable U) iniiiil submission for student data elements verified 
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independent students to have an error (37.1 percent versus 12.8 percent). However, the 
processing edits select dependent students for Integrated Verification only slightly more 
frequently than independent students (38.6 percent versus 30.2 percent). 

Exhibit V-10 also shows that applicants with high AGI are much more likely to have an 
error than those with low AGI. For dependent students, the error rate among parents 
with AGI over $35,000 is more than six times as great as that among parents with AGI 
less than $5,000 (66.6 percent versus 10.4 percent). Similarly, for independent students, 
the error rate among students with AGI over $15,000 is more than seven times as great 
as those with AGI less than $5,000 (34.4 percent versus 4.7 percent). The processing 
edits select dependent students with high AGI less frequently than those with low AGI. 
While independent students with AGI over $15,000 are selected more frequently than 
other independent students, the edits still select nearly one-quarter of the independent 
students with AGI less than $5,000 and between $5,001 and $10,000 - the groups with 
the smallest error rates. 

These findings suggest that ED processing edits could be improved by targeting 
dependent students more frequently than independent students and targeting high AGI 
students and parents more frequently than low AGI students and parents. Our database 
only includes Ti le IV recipients, and while the selection system is applied to all 
applicants, it is not possible for us to fully evaluate the effectiveness of the selection 
system. 
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VL SIMPUFICATION OF THE PELL GRANT PROGRAM 

Simplification of the Title IV Student Financial Aid Delivery Systems has long been a 
concern of many members of the student financial aid community. Over the years, 
ED and other organizations have developed alternative proposals to simplify the 
delivery systems in order to reduce processing costs, burden, and error. Prior quality 
control studies have continuously recommended that simplification of the delivery 
process might help reduce systemic errors while producing rninimai impact on program 



This chapter describes our analysis of a proposal by the National Association of 
Student Financial Aid Administrators' (NASFAA's) Need Analysis Standards 
Committee (NASC) to simplify the TiUe IV financial aid formulas. We modeled the 
effect of the proposed changes to the Pell Grant Program because the effect of the 
changes could be easily traced to changes in award amounts, which is not the case for 
the Campus-Based and Stafford Loan programs. 

This section is organized as follows: 

A. Description of Modifications to the Pell Grant Formula 

B. Effects of Modifications on Dependency Status 

C. Effects of Modifications on Pell Grant Awards 



payments. 
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A. Description of Modifications to tlie Pell Grant Formula 

We modeled a total of eight individual changes to the Pell Grant Program: 

1. Revising the definition of dependency status 

2. Eliminating medical and dental expenses from the formula 

3. Eliminating elementary and secondary tuition from the formula 

4. Eliminating the simplified formula 

5. Eliminating work income from the formula 

6. Making AFDC recipients automatically eligible for a full award 

7. Eliminating assets from the formula for lower income families 

8. Using a calculated value of U.S. taxes paid rather than the reported 
figure 

Below we will describe how we modified the necessaiy components of the Pell Grant 
Formula to model these changes. 

1. Revising the Definition of Dependency Status 

The definition of dependency status used for determining Pell Grants in 1988-89 
included many complicated questions. In addition, many of the answers to these 
questions were hard to verify. In order to streamluie the determination of 
dependency status, NASC developed an alternative definition of dependency status 
under which a recipient was considered independent if they met at least one of the 
following criteria: 

er|c is(; 
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• Were bom before January 1, 1965 (for the 1988-89 award year) 

• Were an orphan or ward of the court 

• Had legal dependents other than a spoiLse 

• Were a graduate or professional student 

• Were a veteran of the U.S. Armed Forces 

Any recipient not meeting at least one of these criteria is considered dependent under 
the revised definition. 

2. Eliminating the Offset for Medical and Dental Expenses from the 
Pell Grant Formula 

Under the 1988-89 Pell Grant formula, total income was offset by the portion of 
medical and dental expenses that exceeded 20 percent of effective family income 
(total income, minus U.S taxes paid, minus the state tax allowance). Because a 
relatively small number of recipients qualify for the medical/dental expense offset, 
many members of the financial aid community view medical/dental expenses as a data 
item that could be eliminated from the Pell Grant formula without significantly 
affecting the distribution of financial aid. For the alternative formula, we set the 
offset to $0 for all recipients. 

3. Eliminating the Offset for Elementary and Secondary Tuition 
from the Pell Grant Formula 

Under the 1988-89 Pell Grant formula there was an offset of up to $3,450 for each 
family member for whom the parents paid elementary or secondary tuition. Like the 
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ofSsQt for medical/dental expenses, a relatively small number of recipients qualify for 
the tuition offset Elementary /secondary tuition paid, therefore, has been seen as a 
data item that could be eliminated from the Pell Grant formula. For the alternative 
formula, we set the offset to $0 for all recipients. 

4« Eliminating the Simplified Fonnula 

The 1988-89 Pell Grant fonnula allowed for the Pell Grant to be calculated using a 
shortened formula in certain iiistances. This shortened formula could be used by 
recipients that: (1) did not have total income (parents and student/spouse) of $15,000 
or more, and (2) did not file an IRS 1040 long form. By making the general 
application simpler, the need for a simplified formula could be eliminated. Any 
special circumstances could still be handled through professional judgment governed 
by appropriate Federal guidelines. For the alternative formula, we eliminated this 
secondary formula. 

5. Eliminating Work Income from the Pell Grant Formula 

Ihe 1988-89 Pell Grant fonnula used income earned firom work in two instances: (1) 
to calculate total income when the student or parents did not file an income tax 
return, and (2) to calculate an employment expense offset. The distinction between 
work income and non-work income is confused by many recipients. Furthermore, for 
many recipients, there is little difference between work income and AGI. For these 
reasons, it has been discussed that work income could be eliminated from the Pell 
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Grant formula. The alternative formula eliminated work income from the formula by 
setting work income to $0. We modified the calculation of the employment expense 
offset to use AGI, not work income. The employment expense offset was calculated 
by assuming that AGI was equal to work income, and that, if thr recipient (or his 
parents) were married, their AGI was split evenly between husband and wife. The 
maximum offset of $1,500 was maintained in the alternative formula. 

6. Making AFDC Recipients Automatically Eligible for a Full Award 

Under the 1988-89 Pell Grant formula, AFDC recipients were not treated any 
differently than non-AFDC recipients. Because students and parents must 
demonstrate significant financial need in order to receive AFDC, there has been 
discussion among members of the financial aid community that the financial aid 
delivery system could streamline the application process for AFDC recipients. The 
alternative formula gave anyone who received AFDC benefits (student or parent) an 
SAI of 0. This modification meant that they were eligible for the maximum Pell 
Grant given their enrollment status and cost of attendance. 

7. Eliminating Assets from Lower Income Families 

Under the 1988-89 Pell Grant formula, assets for ail students and parents were 
analyzed and they were expected to contribute a portion of the available assets. For 
families with relatively low total incomes (less than $25,000), it is reasonable to 
assume that available assets are usually small or non-existent. For the alternative 
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fonnula, we set a family's net available assets to $0 if they met both of the following 
conditions: (1) they filed a 1987 tax return on Form 1040A or Form 1040EZ\ and (2) 
they had less than $25,000 in total income. 

8. Using a Calculated Value of U.S. Taxes Paid Rather Than the 
Reported Figure 

The 1988-89 Pell Grant formula used for U.S. taxes paid the amount reported on the 
application for financial aid. A common error on the financial aid application is to 
report taxes withheld rather than taxes paid. Because of this, there has been 
discussion about calculating U.S. taxes paid based on the reported filing status and 
reported number of exemptions. For the alternative model, we calculated U.S. taxes 
for all recipients and parents based on filing status, number of exemptions, and 
adjusted gross income. The calculated amount of taxes paid wiU not always equal the 
true amount, because the calculated amount cannot take into account the effect of 
itemized deductions and tax credits on taxes paid. 

B. Effects of Modifications on Dependency Status 

This section describes the effect of changing the definition of dependency status, while 
Section C describes the effect of the other seven changes as well as the effect of all 
changes taken together. We describe below what types of recipients are affected most 
by the alternative definition of dependency status and how the alternative definition 
can be modified to reduce the number of changes in dependency status. 

^ Taxpayers who itemize deductions (e.g., to take the mortgage interest 
deduction) or who have certain income sources, adjustments to income, additional 
taxes, or tax credits must file their tax return on Form 1040, rather than on Form 
1040A or Form 1040EZ. 
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1. Effect of Alternative Definition of Dependency Status 

To estimate the effect of the alternative definition of dependency status, we 
determined the dependency status for each recipient using the alternative definition, 
and compared this dependency status to the dependency status determined using the 
cunent formula. Exhibit VI- 1 presents a summary of the results of this comparison. 



Exhibit VI-l 



Effects on Dependency Status of Modifying 
the Dependency Status Definition 



Current 
Formula 


Alternate 
Formula 


Percent of 
Students 


Median 
Student 
Age 


Median 
Student 
Income 


Dep. 


Dep. 


49.6 


20 


$1,836 


Dep. 


Ind. 


0.3 


23 


$4,124 


Ind. 


Dep. 


3.9 


22 


$6,001 


Ini 


ImL 


46.2 


22 


$8,536 


M 


All 


100.0 


22 


$3,889 
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As shown, nearly 96 percent of our sampled recipients had the same dependency 
status under the current and alternative definitions. Only 0.3 percent of recipients 
were dependent under the current formula and independent under the alternative 
formula, and 3.9 percent were independent under the cunent formula and dependent 
under the alternative formula. 

All of the recipients in our sample who switched from Dependent to Independent 
were graduate students who were claimed as dependents by their parents for tax 
purposes (these recipients represent only 6.9 percent of all graduate students 
analyzed). 

The recipients who switched from Independent to Dependent were students who did 
not meet any of the five criteria under the alternative definition but were Independent 
under the cunent formula. As is indicated in the fourth column of Exhibit 31.5 
percent of all switchers were married students who were classified as dependent under 
the alternative definition. Of the married students who would be classified as 
dependent imder the alternate definition, 81.4 percent were classified as independent 
under the current definition. These students had a higher median income than 
students who were not married, including those unmarried students classified as 
independent under the alternative definition. 

Because of the effect on married students, it may be appropriate to modify the 
alternative definition to include married students as Independents. This modification 
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would have a net effect of reducing changes in dependency status. This reduction 
would occur because, as shown in Exhibit VI-2, of the married students who were 
dependent under the alternative formula, 81.4 percent and 18.6 percent were 
independent and dependent respectively under the current formula. Thus, 81.4 
percent of these recipients switch status and 18.6 percent do not. If the alternative 
definition included married students as Independents, 18.6 percent of this group would 
switch status and 81.4 percent would not. This would reduce the overall rate of 
dependency status switchers from 4.2 percent to 3.2 percent. 




Exhibit VI-2 



Dependency Status Switchers: 
Marital Status as a Possible Factor 
in Determining Dependency Status 



STUDENTS CHANGING STATUS 



MARITAL 

5TATV5 


ALTERNATE 


PERCENT^ 

CHANGING 

STATUS 


PERCENT 
OF 

TOTAL 


MEDL\N 

STUDENT 

AGE 


MEDLMM 

STUDENT 

INCOME 


Not Married 


Dep. 


4.9 


60.7 


22 


$5,787 


Not Mai'iicU 


Ind. 


LI 


7.8 


23 


$4,124 


Manied 


Dep. 


8L4 


31J 


21 


S8,921 


Married 


Ind. 


0.0 


0.0 


N/A 


N/A 



1 4.2 p«rc0nt of all students changed dependency status 
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2. Conclusions 

It is clear that the alternative defimtion of an Independent student is simpler than the 
current definition. Relative to the current definition, the alternative definition 
requires fewer questions, does not involve complicated branching, and uses data items 
for which it is easy to obtain verification. Including married students in the 
alternative defijiition could oe accomplished by changing the wording on one of the 
five questions from "Do you have legal dependents other than a spouse?" to "Do you 
have legal dependents?" 

C Effects of the Modifications on Pell Grant Awards 

This section describes the effects that the modifications mentioned previously had on 
Pell award amounts. We first describe the effect on Pell awards when all of the 
changes were implemented simultaneously, and then describe the effect on awards of 
implementing each change by itself. All of the results discussed below are 
summarized in Exhibit VI-3. 

To evaluate the effect of the alternative formula, we calculated Pell awards using the 
alternative formula and compared them to Pell awards calculated using the current 
formula. Our analysis had the following limitations: 

• We could not calculate alternative awards for most of the recipients who 
switched from independent to dependent because we did not have the 
parent information necessary to calculate Pell award amounts. 
Fortunately, these dependency status switchers represent only a small 
portion (4.2 percent) of the recipients. 

• Recipients categorized as dislocated workers or displaced homemakers 
were excluded from our analysis because their award would be 
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Percent of Case« with Differences Between 
Current and Alternative Formula 
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detennined through professional judgment under the alternative formula. 
It was not possible to accurately model how these recipients would have 
been treated through professional judement. As with dependency status 
switchers, dislocated workers and displaced homemakers represent only a 
small portion of the recipients. 

L Changes in Pell Grant Awards 

When all eight of the modifications weic implemented, 79.5 percent of the Pell Grant 
recipients in our data base had a change in their Pell Grant of $50 or less, and over 
95 percent had a change of $250 or less. More Fell Grant recipients had an increase 
in their award under ths alternative formula than had a decrease in their award (13.3 
percent had an increase in their award of at least $51 while 7.4 percent had a 
decrease in their Pell award of at least $51). 

The average award fo: our sample was $1,338 using the current formula and $1,339 
using the alternate formula. For those recipients whose award changed, the average 
decrease ($270) was almost 65 percent larger than the average increase ($164). The 
average absolute change in awards, for recipients whose award changevl, was $197. In 
addition, as shown in Exhibit VI-4, nearly 97 percent of recipients vdth awards over 
$2,000 under the current formula also had awards over $2,000 under the alternative 
formula. Thus, very few of the neediest recipients had a change in their award as a 
result of the alternative formula. 

Using a calculated rather than reported amount for U.S. taxes paid affected a higher 
percentage of Fell Grant recipients than any other single change modeled. When this 
change was implemented just over 7 percent of the recipients had their award change 
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Exhibit VI-4 

Distrijution of Pell Awards 
Under Current and Alternative SAI Formulas 
(Number of Sampled Recipients) 



AWARD AWARD IJNDF.R CURRENT FORMULA 



UNDER 



ALTERNATE 
FORMULA 


$500 


$501 - 
$1,000 


$1,001- 
$1,500 


1,501- 
$2,000 


OVER 
$2,000 




$500 or Less 


155 


13 


2 


1 


1 


172 


$501 - $1,000 


15 


245 


10 


1 


1 


272 


$1,001 - $1,500 


2 


26 


332 


5 


1 


366 


$1,501 - $2,000 


0 


1 


13 


273 


10 


297 


Over $2,000 


0 


0 


I 


US. 


280 


397 


All 


172 


285 


359 


295 


393 


1,504 



by more than $50. The percentage of recipients whose award decreased by more 
than $50 (4.3 percent) was slightly higher than the percentage of recipients whose 
award increased by more than $50 (2.6 percent). There are two reasons a recipient's 
award amouit could have changed: 

• The recipient reported the correct amount of taxes paid and, because 
the calculated amount cannot take into account tax credits, the 
calculated amount was incorrect. 
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The recipient reported the incorrect amount of taxes paid (a common 
mistake is reporting taxes withheld) and the calculated amount was 
different than the reported amount. 



When the Pell formula was modified for AFDC recipients, eliminating assets for low 
income recipients, and eliminating work income, there were veiy few changes in 
awards. For each of these three changes, less than 2 percent of recipients had their 
Pell award change by more than $50. In addition, for each change, a higher number 
of recipients had their awards increase than decrease. 

The modifications for medical/dental expenses and elementary and secondary tuition 
had similar effects on Pell Awards. These changes decreased awards for 2.6 percent 
(elementary and secondary tuition) and 3.2 percent (medical/dental expenses) of 
recipients. No awards were increased. 

It is important to note that the percentage of students whose award changed when all 
modifications were modeled (20.5 percent of recipients) was higher than the sum of 
the percentages of recipients whose award changed under each individual modification. 
This pattern could occur for several reasons, including: 

• The effect of the changes is cumulative. That is, a recipient's award 
could have decreased by $40 when medical/dental expenses was 
eliminated and by $40 when elementary and secondary tuition was 
eliminated. This would have lead to changes of $50 or less under the 
individual changes but a combined change of more than $50. 
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2* Conclusions 

It is clear from the results presented that simplification of the Fell Grant formula can 
be accomplished with relatively minor effects on most recipients and on aggregate 
awards. However, even changes that have only small effects on awards imply a series 
of equity issues which go beyond the scope of this study. For example, while 
relatively few recipients are affeaed by eliminating an allowance for elementary' and 
secondary tuition, does the Department want to treat a family with elementary and 
secondary tuition expenses the same as a family with identical circumstances except 
that they do not have elementary and secondary tuition expenses? Only the Congress 
in conjunction with the Department and the financial aid community can weigh the 
equity issues inherent in ultimately adopting a package of changes to simplify the 
delivery system. 

ED, Congress, and the financial aid community agree that there is a need for 
simplification of the financial aid delivery system. The current complexity of the 
financial aid delivery system is a result of many previous attempts to address error and 
equity through changes to the financial aid formula. The result of this process is a 
formula that is burdened by the need to account for issues that affect only a small 
handful of applicants. A simplified process that handles the majority of situations, 
coupled with the use of professional judgement within guidelines provided by ED, may 
be an answer to the growing complexity of the system. 




Vl-15 

199 



m CORRECTIVE ACTIONS 



lOCMP Findings and Corrective Actions 



A. Introduction 

This section presents corrective actions for ED to consider in its efforts to reduce 
error in the Title IV student financial assistance programs. These recommendations 
are based on both the findings of the IQCMP and on recognized management 
practices used by other organizations within and outside of the Federal government. 
Because the purpose of IQCMP was to examine error in the Title IV programs, most 
of the following suggested improvements are aimed at reducing error. 

1. Frameworlc for Corrective Actions 

Corrective actions with regard to the Title IV delivery system should be aimed 
towards increasing the quahty of services provided by ED. There are three distinct 
groups served by ED: students who are potential recipients of Title IV aid (and their 
families), educational institutions that participate in the program, and taxpayers who 
pay for the program. Each of these three groups has specific goals that must be 
addressed by ED. ED can increase service to students and institutions by increasing 
the number of services available, decreasing turn-around time, providing clearer and 
more in-depth information, and reducing the burden involved in the financial aid 
system. At the same time, ED needs to ensure that taxpayer money is being spent 
correctly (i.e., with a minimum of errors), equitably, and in a cost-effective manner. 

Because of the complex nature of the student aid delivery system, errors in awarding 
student financial assistance will never be entirely eliminated. Some reduction in error 
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rates could be achieved by implementiiig minor modifications to the delivery system 
(e.g., improving instructions, redesigning forms, etc), but many of these "quick fixes'* 
have been tried in the past with limited success. ED may need to consider more 
sweeping changes in the financial aid delivery system, such as simplification of the 
financial aid formula, in order to achieve major reductions in error rates. 

« 

2« Causes of Error 

Both students and institutions have motives to maximize the student's Federal aid 
awards. Students have the obvious motivation of reducing the financial burden of the 
costs of postsecondary education. Institutions also benefit from Federal student aid, 
either through reduced demands on institutionally funded financial aid or through 
increased enrollment by financially needy students. These pressures to increase 
Federal aid awards may lead to intentional errors. Intentional errors include: 

• DeUberate misstatements of fact, 

• Stretching the truth to a more favorable outcome, and 

• Intentional failure to report a change in the student's situation. 

Not all of the error in the Title IV programs Cin be attributed to intentional errors. 
Most students and institutions are honest and intend to supply complete and accurate 
information, but mistakes are still made. Unintentional mistakes can effect awards 
either in a random fashion (e.g., an unintentional error in home value could increase 
or decrease the award) or in one direction (e.g., forgetting to report untaxed income 
will increase the award). Overall, unintentional mistakes probably tend to increase 
the student award. 
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Unintentional enors include: 



• Confusion over policies and procedures (e.g., failure to report changes in 
situation), 

• Incorrect estimation of data elements (e.g., an estimate of home value 
made without an appraisal), 

• Incorrect calculations performed (e.g., an addition error is made when 
calculating other financial aid awaraed), or 

• Confusion caused by timing problems (e.g., failure to include an outside 
award that was made after the Federal aid was packaged). 

Although ED has uncovered several cases of fraud in recent years, it is nearly 
impossible for a study such as IQCMP to differentiate between the effects of 
intentional and unintentional errors. We uncovered only one case that was clearly an 
example of fraud, that of a student's submission of a falsified application for financial 
aid. For this reason, the corrective actions presented in this report are designed to 
reduce both intentional and unintentional error. 

3. Corrective Action Strategies 

Five different strategies for reducing error in the student assistance programs are 
discussed in this chapter. These five strategies are intended to cover both intentional 
and unintentional errors committed by students «ind institutions. The five strategies 
include: 

1. Improve communications with students and institutions » This corrective 
action strategy, directed only at unintentional errors, has been the focus 
of many past corrective actions (e.^., clarification of instructions about 
household size was a recommendation of the Stage II study). This 
strategy would be relatively easy for ED to implement, but would 
probably not result in large reductions in error. 
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2. Remove opportumties for error ~ This corrective action strategy, which 
is targeted at intentional and unintentional institutional and student 
errors, should help to prevent errors before they occur. Simplification of 
die financial aid tormma would be part of this strate^, since there are 
less items in a simplified formula to be in error. Hiis strategy requires 
the most changes to the delivery system, but is also likely to be the most 
successful in making significant reductions in error rates. 

3. Provide disincentives for noncompliance - This corrective action strategy 
should reduce intentional errors since its goal is to increase the 
perception of likelihood of being caught and punished. However, this 
strategy may also reduce unintentional error by making applicants more 
careful when completing their applications. An example of a corrective 
action using this strategy would be to impose more frequent fines on 
students who provide incorrect information on Uieir application. 

4. Identify and correct errors after occurrence - This corrective action 
strate^ is already being employed by ED and includes the Integrated 
Verification program and program reviews. This strategy helps decrease 
intentional and unintentional errors. 

5. Enlist educational institutions in efforts to reduce error - lliis corrective 
action strategy has already been implemented by ED as the Institutional 
Quality Control Pilot Project Many institutions, inside and outside of 
the Pilot project, already have conmiitments to quality control but may 
need closer guidance from ED. 



These corrective action strategies, and proposed corrective actions, are discuss(',d 
below. 



B. Improve Communications With Students and Institutions 



This corrective action strategy involves clarifying instructions to both students and 
institutions, providing students with more information on policies and procedures 
dealing with financial aid, and correcting incorrect perceptions on the part of studei^ts 
and Institutions. 
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This corrective action strategy is relatively easy and low-cost to implement Only 
minor changes in current ED procedures are involved. These corrective actions 
should be undertaken in order to increase quality in the Title IV programs. However, 
because of their nature, it is likely that the changes proposed imder this strategy will 
have only small effects on error in awarding aid. 

Three specific corrective actions that fall within the overall strategy of improving 
communications with students and institutions are discussed below. 

1. Qarify Definitions of Student Reported Items 

This corrective action involves rewriting the definitions of items on the financial aid 
application to remove ambiguity. For example, the definition of household size on 
the 1988-89 A pplication for Federal Student Aid reads, "The number of people that 
you will support between July 1, 1988 and June 30, 1989. Include yourself and your 
spouse. Include your children if they get more than half of their support from you. 
Include other people only if they now live v^th and get more than half of their 
support from you and will continue to get this support between July 1, 1988 and June 
30. 1989." 

The word "support" is ambiguous and could be interpreted as including only the 
necessities of food, clothing, and housing or as including luxuries such as a new car 
and a European vacation. Tha applicant is free to interpret "support" to his or her 
benefit. The definition could be improved as in the example on the next page. 
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Although the word support is defined in the application for 1990-91 to be "money, 
gifts, loans, housing, food, clothes, car, medical and dental care, payment of college 
costs, etc.", this definition appears only in the section on parents' household 
information and will not be read by independent students. The definition could be 
improved as in the following example. 



Example of a Revised Definition for Student's Household Size 

The number of people that you will support between July 1, 1988 and June 30, 
1989. Include yourself and your spouse. Include your children if you provide 
more than half of the money for food, clothing, sheher, medical expenses, 
transportation, and education. Include other people only if they now live with 
you and receive more than half of the money for these items from you and 
will continue to receive this support between July 1, 1988 and June 30, 1989. 



The definition of displaced homemaker status for parents of dependent students is 
ako vague. The 1988-89 A pplication for Federal Student Aid instructs the applicant 
as follows: 

"Qieck 'Yes' if either of your parents meets all of the following descriptions for 
a displaced homemaker: 

• your parent has not worked in the labor force for a substantial number 
of years (e.g., approximately five years or more) but has, during those 
years, worked in the home providing unpaid services for family members; 

• your parent has been dependent on public assistance or on the income 
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of another family member but is no longer receiving that income, or 
your parent is receiving public assistance because of dependent children 
m the home; and 

• your parent is unemployed or underemployed and is experiencing 
difficulty in obtaining or upgrading employment." 

Although it is clear that the applicant's parent must meet all three conditions to be 
considered a displaced homemaker, the conditions themselves contain obscure 
language and may confuse the applicant. For example, the first condition requires the 
parent to have been out of the labor force for a "substantial" number of years, and 
then defines this period as "approximately five years or more". Specifying a fixed 
period of unemployment will reduce confusion on the appUcant's part which will lead 
to a reduction in unintentional errors. 

The second condition requires parent dependence on public assistance, but fails to 
define what constitutes public assistance or what it means to be dependent upon 
public assistance. We suggest a reduction in unintentional errors can be attained by 
listing examples of eligible public assistance programs (e.g., AFDC, welfare, Social 
Security, etc.) and defining dependence as a fraction of family income (e.g., 
"dependent" means that at least 50 percent of available family resources comes from 
public assistance). 

Finally, the tliird condition refers to "unemployment" which is defined in the 
instructions as follows: 

"Unemployment" means not working this week but being available for work and 
having made specific efforts to get a job sometime during the last four weeks. 
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In this context, "specific effi rts" may include anything from glancing at the classified 
adds to listing with a job placement agency. "Specific ei^orts" should be further 
defined according to the standards implemented by the Bureau of Labor Statistics. 
BLS defines "specific efforts" to obtain employment as follows: 

• place or answer classified advertisements in the newspaper 

• contact a potential employer directly 

• contact friends or relatives about obtdning employment, or 

• contact a public or private employment agency. 

Specifying precise definitions throughout the instructions will reduce applicant 
confusion and thus reduce the opportunity for unintentional error. Before the 
financial aid application is released, it should be reviewed by a trained editor. Focus 
group sessions with student and parents could be used to obtain additional input on 
the clarity of the questions and instructions. 

2. Clarify Procedures For Reporting Changes in Student Situation 

On the second page of the financial aid application the student is told that if his or 
her situation changes pertaining to questions 3-3 and 3-4 (parent's household size and 
number in college) or Step 4 (student's household size and number in college) to 
"wait until you receive your SAR and then see your financial aid administrator." This 
wording seems to indicate that notifying the financial aid administrator is optional. 
To address this issue, the following sentence could be added to the application by the 
questions about household size and number in college: " NOTE : If your situation 
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changes before you receive an award, you must notify your financial aid 
administrator". 

3. Clarify Regulations Regarding the Student Assistance Programs 

During IQCMP, two areas were uncovered in which ED has given the institutions 
unclear or erroneous instructions about the policies involving the Title IV programs. 

These two areas involved Stafford Loan undercertifications and the worksheets used 
by the institutions to calculate family contributions. 

Under the original analysis plan proposed for IQCMP, undercertifications in the 
Stafford Loan program were to be included as errors. However, information received 
from institutions during the data collection phase and during foUow-up conversations, 
indicated that there v/as a considerable amount of confusion over whether a student 
who requested a loan smaller than the amount for which he or she was eligible 
should be certified for the smaller loan or for the full amount. Queries to ED 
produced the same ambiguous responses, and the matter is not discussed in the 
financial aid handbooks. Gearly, ED needs to clarify its policy regarding Stafford 
Loan undercertifications and notify the institutions. 

The Pell Grant and Congressional Methodology handbooks contain worksheets that 
may be used by the institutions to calculate the student's contribution to the cost of 
education. These worksheets are used either to recalculate a contribution after 
changing data or to estimate contributions until the official SAR is received. Since 
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the family contribution is an essential ingredient in calculating student awards, it is 
very important that the institution be able to complete these calculations accurately. 
However, in programming the family contribution formulas for use in IQCMP, errors 
were found in these worksheets. ED needs to be certain that these worksheets are 
correct and that educational institutions understand how to use them. 

C Remove Opportunities for Error 

This corrective action strategy is aimed at reducing errors before they have an 
opportunity to occur. Corrective actions that could be implemented under this 
strategy involve eliminating data required to compute the financial aid award, 
requiring data to be supplied following specific formats or at specific times, and 
simplifying the delivery system. 

The corrective actions suggested as part of this strategy require more substantive 
changes in the deUvery system than do the other strategies. Because this strategy 
should eUminate opportunities for error, the error rates should decrease. 

Three specific corrective actions that are designed to reduce opportunities for error 
are discussed below. 

1. Simplification of the Financial Aid Formula 

This corrective action, which has been discussed for several years in various forms, 
will involve a major investment in an overhaul of the financial aid delivery system. 
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Steps needed to implement a simplified lormula include: 

• Obtain legislation to authorize changes to the Pell SAI and 
Congressional Methodology formulas. 

• Change the programs used to compute the family contribution formulas. 

• Train financial aid administrators on differences in the formula. 

• Revise ED policies and regulations to follow simplified formulas. 

• Redesign ED publications including the financial aid application to 
reflect the simplified formulas. 

Despite the initial costs involved in implementing a simplified formula, the potential 
for later savings may make simplification worthwhile. Potential benefits to be derived 
from simplification include a reduction in error rates, a decrease in the burden 
imposed on ?^oth students and institutions in ttie award process, and possible savings 
in the cost of processing and verifying student applications. Further study is needed 
to determine precise estimates for the savings potential of a simplified formula. 

Chapter VI of this report discusses a simplified student aid formula proposed by 
NASFAA. This formula will simplify the aid process for many students, especially the 
neediest, while leading only to small redistributions in the aid awarded. Errors 
caused by those items eliminated from the formula, (e.g., work income, medical/dental 
expenses, and assets for most families with incomes less than $25,000, etc.) will be 
removed. 

In addition to error reduction, this formula may increase the number of awards to the 
neediest students. Because those receiving public assistance will receive a full award 
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under the proposed formula, they may be encouraged to apply for aid. Estimating 
the number of additional students who would apply for aid imder these circumstances 
is beyond the scope of IQCMP. 

2. Revise Reporting Procedures for Pell Cost of Attendance 

During the processing of IQCMP data, several cases were found to be in error 
because the institution failed to apply the regulatoiy Umits to the components of cost 
of attendance in the Pell program. Because the rost of «ittendance is usually reported 
as one lump sum, the total cost of attendaz^'* could be within the prograo ceiling but 
still be too high if one component is higher than the regulatory cap. 

lliere are two alternative corrective actions that could be used to eliminate this error. 
The first would be to require the schools to report each item within the cost of 
attendance separately, thus making a component that is above the limits obvious. The 
second option would be to eliminate the ceilings placed on the components of Pell 
cost of attendance. The disadvantage to this solution is that either the Pell awards 
would increase (i.e., student need would be greater because cost of attendance is 
higher and the family contribution remains the same) or the cost of attendance would 
have to be rescaled in order to hold Pell awards stable. Both of these corrective 
actions should significantly reduce cost of attendance errors. ED should decide 
between these alternative actions based on its poUcy goals. 

3. Do Not Disburse Awards Based on Estimated Tax Data 

As reported in the error profile analysis discussed in Chapter IV, filing a financial aid 



ERIC 



VIM2 

211 



/ 



lOCMP Findings and Correctivg Actions 

application based on estimated tax data was linked to significantly increased student 
error. Although a student tnay need to file a financial aid application before a tax 
return is complete, the student usually will not receive disbursed funds until after a 
tax form is filed. We recommend that funds not be disbursed until after a student 
updates estimated tax data. 

There are two possible methods for receiving updated tax data. The first is to send a 
letter to the student in late April which lists the student's estimated tax data. The 
student would be required to update the estimated tax data with correct tax 
information or to sign a statement certifying that the estimated tax data is correct. 
This method would place the burden of updating tax information on the student and 
would fit in with ED's traditional poUcy of relying on applicant integrity. 

The second alternative for receiving upcated tax information is to require that ail 
students who use estimated tax data send a copy of their completed tax return to the 
postsecondary institution. Many students who used estimated data would be likely to 
update their SAR since the tax form would be available to the institution; however, 
the burden of verifying the tax data would lie with the institution. This method 
would likely lead to a greater reduction in errors than the previous method but would 
also be more burdensome to the institution. 

No matter which of the above two methods is used to collect updated tax information, 
the award should not be disbursed until the updates are received by the awarding 
institution. This process could work similarly to that used to ensure that a financial 
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aid transcript is in the file of all transfer students. For special cases where the 
student should receive money before the tax return is filed, the institution could make 
one disbursement. 

D. Provide Disincentives for Noncompliance 

This corrective action strategy involves creating the perception that data will be 
checked and that the potential for stiff penalties more than offsets the benefits of 
error. To be fully successful, this strategy requires a delivery system that holds both 
students and institutions accountable for error and that uses only data elements that 
can be verified. 

This strategy is primarily aimed at reducing intentional errors. Applicants who 
deliberately report incorrect information will not be as likely to do so if they perceive 
that they will be caught and punished. A secondary benefit of the strategy is 
potential reduction in unintentional errors because people will be more careful in 
completing the application if they know they will be penalized for an error. 

Specific corrective actions that implement the strategy of providing disincentives for 
noncompliance are discussed below. 

L Require Specific Information on Household Size and Number in 
College 

As reported in Chapter III, household size and number in college contribute 
significantly to student errors uncovered during IQCMP. While some of these errors 
are undoubtedly due to projecting error (i.e., students were asked to project the 
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household size and the number in college for the upcoming academic year), others 
are probably due to inaccurate reporting of the household size or number in college, 
either inadvertently or deliberately. Previous studies have suggested that changes in 
the family's circumstances is not a primary reason for errors in these two questions. 

The IRS has had a similar problem with the number of dependents that people claim 
on tax forms. To correct this problem, the Service currently has tax filers complete a 
table that lists each dependent's name, social security number, and months in the 
household. The IRS has found that this information has reduced the number of 
dependents claimed in error, despite the fact that the data is seldom validated. 

ED should use a similar table on the application to solicit information on household 
size and number in college. ED has considered this approach in the past, and 
rejected it due to cost considerations and limited space on the application. However, 
we believe the addition of a household size/number in college table will significantly 
improve the quality of application data, and should be reconsidered by ED. 

An example of a table that could be used to collect information on an independent 
student's household size and number in college is shown on the following page. This 
table could also be used to track information during the verification process. 
Although it is unlikely that an institution will use the information on the form for 
purposes other than verification, applicants will understand that the institution could 
use this information and may complete their application more carefully. 
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Name 



Relationship Claimed as Enrolled in Institution 

to Student Dependent on College Attending 
Tax Return 

(Yes or No) (Yes or No) 



1. 


SELF 




YES 




2. 


SPOUSE 








3. 










4. 










5. 










6. 






1 





Total Number in Household from Above: 
Total Number in College from Above: 



2. Require Specific Information on Home Value and Debt 



As discussed previously, home value was found to be a leading source of student error 
in the Title IV programs. Home value is a difficult item to verify because there is 



often a lack of documentation on the real value of a home. Applicants probably tend 
to understate, either intentionally or unintentionally, the value of their family's home. 
Wc are proposing that ED collect specific information on home value and debt 
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including: 

• Current home value, 

• Current home debt, 

• ZIP code of home, 

• Original purchase price of home, and 

• Year of purchase. 

Information on home value could be scanned by ED or the institution for 
reasonableness using figures on the local real estate market indicated by the ZIP 
code. These figures could be obtained from local governments or private sources. 
This would be a time co-isuming process and would probably not be worthwhile unless 
a student is selected for verification or is suspec(:ed of error. However, we 
recommend that this information be included on the application because of the 
perception that home value could be checked. This perception might be enough to 
eliminate many reported inaccuracies in home values. 

A pending congressional action may eliminate home value from the financial aid 
formula. In addition, several reauthorization proposals that are currently being 
considered are recommending deletion of home value because it is not a true liquid 
asset that can be used to pay for the costs of education. If home value is not 
eliminated, then ED should determine whether the added burden of collecting and 
using this information is offset by the savings from error reduction. 

3. Apply Penalties More Frequently to Cases With Error 

The A pplication for Federal Student Aid warns applicants that if they receive 
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assistance based on incorrect data they will have to pay it back and "may also have to 
pay fines and fees." The application continues on to warn, "If you purposely giv;; 
false or misleading information on your appUcation form, you may be fmed $10,000, 
receive a prison sentence, or both." However, it appears that these penalties for 
giving faulty information, either intentionally or unintentionally, are seldom applied. 
Therefore, the student has no incentive to not '^stretch the truth" about his financial 
situation because if he is verified ^ d loses the aid, he will be no worse ofi than if he 
had provided the correct information in the first place. 

ED should begin using fines more often for student applications with incorrect data. 
Applying small penalties to cases with intentional error will reduce future error by 
increasing the perceived likelihood of being caught. Although students probably do 
not view the postsecondary institution as an enforcement entity, and although 
enforcement is not a primary mission of ED, the perception of a significant ED 
enforcement presence is necessary to promote voluntary compliance by students. 

E. Identify and Correct Errors After Occurrence 

Tms corrective action strategy involves verifying data that has already been collected 
and ensuring that proper ED procedures are being followed by the institutions. The 
corrective actions in this strategy are relatively higher in cost since they involve 
personnel to check and correct past actions, instead of eliminating errors before they 
occur. However, it is important that ED maintain an oversight presence for 
institutions and students. 
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The Department of Education has already im plemented corrective actions using this 
strategy. Integrated Verification focuses on Pitching student errors, and an 
institutional auditing and review policy focuses on uncovering institutional errors. 
However, as discussed in Chapter V on verification, it appears that targeting students 
for verification could be significantly improved. A full review of verification 
procedures was beyond the scope of IQCMP. 

F. Enlist Educational Institutions in Efforts to Reduce Error 

Many instimtions already have a strong commitment to reducing error in the Title IV 
programs. Most institutions (over 84 percent) verify all ED-selected applicants, and 
almost 73 percent of institutions have institutional verification policies (i.e., students 
are verified who where not selected by ED). Nearly 43 percent of the institutions 
performing institutional verification say that they verify all students. This commitment 
to quality control should be encouraged by ED. 

Quality control is implemented in the Title IV delivery system by assessing penalties 
for errors that are made by the institutioas. There are few incentives in place to 
encourage the institutions to improve the quality of financial aid delivery beyond a 
level that is needed to avoid penalties. Under the current regulations, all schools are 
treated equally, regardless of their past performance as measured by error rates. 
Institutions that understand and apply quality control principles, and thus achieve 
lower error rates, may be spending too much time fulfilling regulations; some of this 
time might be better spent in implementing error reduction procedures geared toward 
their own institution. In order for a program such as this to be a success, monitoring 
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of the error rates for these institutions would be needed to ensure that quality control 
practices are not slipping. 

One promising program that involves institutions in efforts to reduce error is the 
Institutional Quality Control Pilot Project. This project is still a relatively new 
project. It has been successful in recruiting instiUitions to participate, but reduction in 
error rates have not been as large as desired. The Pilot project represents a 
promising quality control program focusing on management practices and should be 
refined and continued. 

Institutions that are not approved for, or that choose not to participate in, the Pilot 
project should also receive ED guidance in using quality control procedures to 
improve financial aid management. ED, through its training and dissemination offices, 
could provide a series of brochures or booklets on quality control procedures that 
institutions, especially those in the Pilot project, have successfully implemented. In 
addition, ED should continue to encourage institutions to develop quality control 
procedures and adopt procedures found to reduce error at other institutions. ED 
could promote commitment to quality control through its "Dear Colleague" letter and 
encourage schools to network with each other through NASFAA and other 
professional organizations. 

G. Conclusions 

The five corrective action strategies discussed above require different amounts off 
effort for implementation and will require minor to wholesale changes in the delivery 
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system. Exhibit VII-l presents a summary of the extent of the modifications to the 
delivery system required to implement each corrective action strategy. The corrective 
actions suggested under each of these strategies can be implemented in any 
combination that is deemed appropriate to meet the Department's goals. 

Despite the decrease in error rates found during ICXJMP, error in the Title IV system 
is still significant. ED should review its goals for quality control, as well as the 
corrective actions discussed above, and develop a plan for reducing error rates 
consistent with its goals for providing quality service. 
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EXTENT OF MODMCATIONS TO THE TITLE IV DELIVERY SYSTEM 
REQUIRED BY CORRECTIVE ACTION STRATEGIES 



Corrective Action Strategy 

Improve Communications With 
Students and Institutions 

Remove Opportunities for Error 

Provide Disincentives for 
Noncompliance 

Identify and Correct Errors 
After Occurrence 

Enlist Educational Institutions in 
Efforts to Reduce Error 



Extent of Modifications 



MinoE 



Major 



X 



X 



L 
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Student marginal error refers to the error caused by the student misreportmg an 
individual application item. Student marginal errors were calculated for three 
subpopulations: dependent students, independent students, and all students. The 
subpopulation used was determined by the group of students that had a value for that 
application item. For example, the marginal error for Student Adjusted Gross Income 
was calculated for all students, but marginal error for Parent Adjusted Gross income was 
calculated for dependent students only. Similarly, marginal enor for Student Number in 
College was calculated for independent students only. 

All student marginal errors were calculated using a $50 tolerance, and measured for 
their effect on Pell absolute award, Campus-Based need, and Stafford Loan 
overcertification. 

Chapter III presents significant student marginal error. This appendix presents a 
complete list of each application item's contribution to student error, listed by program 
in the following order: 

• Exhibit A-1 : Marginal error in the Pell Grant (dependent students 

only) 

• Exhibit A-2 : Marginal error in the Pell Grant (independent 

students only) 

• Exhibit A-3 : Marginal error in the Pell Grant (all students: 

dependent and independent) 
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Exhibit A4 : Marginal error in Campus-Based need (dependent 

students only) 

Exhibit A-S : Marginal error in Campus-Based need (independent 

students only) 

Exhibit A-6 : Marginal error in Campus-Based need (all students: 

dependent and independent) 

Exhibit A-7 : Marginal error in Stafford Loan certification 

(dependent students only) 

Exhibit A-8 : Marginal error in Stafford Loan certification 

(independent students only) 

Exhibit A-9 : Marginal error in Stafford Loan certification (all 

students: dependent and independent) 



STUDENT MARGINAL ERROR-PELL AWARD ERROR 
RanllDBd by Frequency and Dollar Impact 
DEPENDENT STUDENTS ONLY 



Exhibit A-1 
Paget of 2 



APPUCATION ITEM 



PERCENT OF 
RECIPIENTS 
WITH ERROR 



PERCENT OF 
DOLLARS IN 
ERROR 



MEAN ERROR 
PER RECIPIENT 
WITH ERROR (S) 



Pareoto' Household Size 

Parent' Number io Cdlege 

PcreoU' Home Value 

Pareots' Taxea Paid 

Pfiento' 1987 Income 

ParenU' 1987 Other Untaxed 
Income and Beoefita 

Pareirts' Other Rcal/Inveitmeat Value 

Mother's 1987 Income From Work 

PareoU' 1987 Social Security BenefiU 

Pareou' Home Debt 

Father*! Expected 1988 Income From Work 
Pareata' 1987 Medical and Dental Expenses 
Parento' 1987 Education Expense 
Parento' 1987 Child Support Received 
Parents' Business and Farm Value 
Parents' Cash, Checking, Savings Value 



15.5 
10.3 
6.6 
3.7 
2.9 
1.9 

1.5 
1.4 
1.4 
1.4 

1.2 
0.9 
0.7 
0.3 

0.1 

C.l 



3.4 
3.6 
2.3 
0.6 
1.2 
0.4 

1.7 
0.3 
0.3 
0.2 
0.3 
0.1 
0.1 
e 

0.2 
0.1 



283 
448 
444 
202 
529 
248 

611 
290 
249 
178 
296 
129 
186 
104 
1650 
249 



* Leu than 0.05% 



224 
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Exhibit A-1 
Page 2 of 2 



APPUCATION ITEM 



PERCENT OF 
RECIPIENTS 
WITH ERROR 



PERCENT OF 
DOLLARS IN 
ERROR 



MEAN ERROR 
PER RECIPIENT 
WITH ERROR (S) 



Piicati' Diqilaoed Horoenuker Status 
Paicau' Marital Status 

Pareoto' Expected 1988 Other Taxable Income 

Father's 1987 Income Front Work 

ParenU' 1987 AFDC or ADC 

Parents' Other Real Estate/ 
Invcatment Debt 

Mother's Expected Income From Work 

Pareau' 1987 Filing Status 

Parents' 1987 Itemized Deductions 

PareaU' 1987 Exemptions 

Age of Older Parent 

Parents' State Of Legal Residence 

Parents' Expected 1988 Untaxed 
Income and Benefits 

ParenU' 1987 Chfldren With 
Education Expense 



0.1 
0.1 
0.1 
0.1 
0.1 
0.1 

0.1 



900 
100 
SCO 
300 

m 

316 

soo 



• Less than 0.05% 
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Exhibit A-2 



APPUCATION FTEM 



PERCENT OF 
RECIPIENTS 
WITH ERROR 



PERCENT OF 
DOLLARS IN 
ERROR 



MEAN ERROR 
PER RECIPIENT 
WITH ERROR (S) 



Studeot't HouMbold Sra 

Student's Number in College 

Student's (& spouse's) 1987 Medical 
and Dental Expenses 

Student's (& spouse's) 1987 
Education Expensea 

Student's (& spouse's) 1987 
Childfea with Education Expeatc* 

Student's Veterans Contributoiy 
BeneTiU Per Month 

Student's Veterans Contributory 
Beneflta Number of Months 



2.9 



1.4 



1.1 



0.S 



S76 



S21 
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Rukod by Frequcocy and Dollar Impact 

ALL STUDENTS (DEPENDENT AND INDEPENDENT) 



Exhibk A-3 
Page 1 of 3 



APPLICATION ITEM 



PERCENT OF 
RECIPIENTS 
WITH ERROR 



PERCENT OF 
DOLLARS IN 
ERROR 



MEAN ERROR 
PER RECIPIENT 
WITH ERROR (S) 



Student's (aad Spouae's) Caih, Savinga, Checking 

Student's (and Spouae'a) Income 

Studeat'a (and Spouae'a) 1987 Other 
Untaxed Income and Betteflu 

Studeat'a (and Si^xue'a) 1987 Taxes Paid 

Student'a 1987 Work Income 

Studeat'a (& spouae'a) Other Real 
Eatate/Inveatroeot Value 

Candidate For Bacbelor'a Degree 

Legal Dependent Other Than Spouse 

Ward of Court 

Studeat'a Expected 01 Bill-DoUara 

Student'a (& qwuae'a) Home Value 

Student's (St qxMiae'a) Expected 1988 
Other Taxable Income 

Year in College 

Student'a (St spouae'a) Dislocated Worker 
Status 



3.1 
1.6 
1.5 

0.9 
0.7 
0.5 

0.4 
0.4 
0.4 
0.4 
0.4 
0.3 

0.2 
0.2 



0.5 
0.9 
0.5 

0.1 
0.2 
0.3 

0.3 
0.2 
0.2 
0.1 
0.1 



0.3 
0.1 



205 
786 
487 

139 
290 
729 

1332 
589 
919 
576 
434 
110 

1569 
1569 



Spouse's Expected 1988 Income From Work 



0.2 



342 



•Less than 0.05% 
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Rtaked by FnquMicy tad DoUir ImfMCt 
ALL STUDENTS (DEPENDENT AND INDEPENDENT) 



Exhibit A-3 
Page 2 of 3 



APPUCATION ITEM 



PERCENT OF 
RECIPIENTS 
Wrni ERROR 



PERCENT OF 
DOLLARS IN 
ERROR 



MEAN ERROR 
PER RECIPIENT 
WTTH ERROR (S) 



Student's Expected Income From Work 0.2 

Student's (tt spouae's) Expected 1988 0. 1 
Untaxed Income And Bcneflu 

U.S. Armed Forces Veteran 0. 1 

Spouae's 1987 Income From Work 0. 1 

Student's (& qxNiae's) 1987 Child Support Received 0. 1 

Student's {St qwuae's) 1987 Social Security BenefiU 0. 1 

Student's Expected 01 BiU-Months 0. 1 

Student's (& qwuse's) 1987 AFDC or ADC 0. 1 

Student's (Sl spouse's) Home Debt 0. 1 

198S Total Rewwrces Of $4,000 or More 0. 1 

Student's State of Legal Residence 0. 1 

Student's Marital Status * 

Student's (& spouae's) Displaced * 
Homemaker Status 

1987 Total Resources Of S4,000 or More * 

U.S. Citizen * 

Student's (& spouse's) Other Real * 
Estate Debt 



214 
148 

183 
347 
122 
388 
147 
100 
200 
53 
1700 
* 

169 



* Less than 0.0S% 
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APPLICATION ITEM 



PERCENT OF 
RECIPIENTS 
WITH ERROR 



PERCENT OF 
DOLLARS IN 
ERROR 



MEAN ERROR 
PER RECIPIENT 
WITH ERROR (S) 



Student*! {Sl qxMiae*!) 1987 ExempCiont 



Received Federal Studeat Aid 1987-88 

Claimed aa Exempbona on PareaU 

1987 looome Tax Return 

Claimed aa Exenptiooa oo ParenU 
1986 Income Tax Return 

Claimed aa Exemptioiio on Pareati 

1988 Income Tax Return 

1986 Total Reaourcea Of $4,000 or More 
Student*! Date Of Birth 

Student*! (& qxMiae*!) Buaineaa and Farm Value 
Student*! (Sl qxMiae*!) Buaineaa and Farm Debt 
Student'! Itemized Deduction! 



^ Lc!! than 0.05% 



Student*! (& qmuaa'a) Filing Statua 



142 
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STUDENT MARGINAL ERROR-CAMFiS-BASED NEED ERROR 
Raakod by ¥n^fmcy and t oUw Impact 
DEPENDENT STUDENTS ONLY 



APPUCATION ITEM 



PERCENT OF 
RECIPIENTS 
WITH ERROR 



PERCENT OF 
DOLLARS IN 
ERROR 



MEAN ERROR 
PER RECIPIENT 
WITH ERROR ($) 



PaicQU' Houaebold Size 

Pucnt' Number in Cdlege 

Panna' Home Value 

Parente' 1987 Medical and Dental 
Ejqpeoaea 

Fatben' 1987 Income From Work 

Pareau' Otber Real/Invettment Value 

Parcou' Home Debt 

Parenu' 1987 Income 

Pareau' 1987 Taxea Paid 

Mother*! 1987 Income From Work 

Parenta' Caih, Checking, Savings 
Value 

Age of Older Parent 

Pareau' 1987 Other Untaxed Income 
AndBeneroa 

Parenu' 1987 Child Support 
Received 



10.6 
8.3 
7.7 
4.5 

4.3 
4.0 
3.9 
3.7 
3.6 
3.2 
2.5 

2.2 
1.8 

0.8 



1.9 
2.6 
1.0 



0.3 
0.4 

1.1 
0.2 
0.2 



0.2 



0.1 



705 
I23S 
489 
235 

256 
396 
340 
1145 
230 
237 
268 

113 
435 

681 



* Leu than 0.05% 
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STUDENT MARGINAL ERROR-CAMPUS-BASED NEED ERROR 
Ranked by Frequeocy aad Dollar Impact 
DEPENDENT STUDENTS ONLY 



APPUCATION ITEM 



PERCENT OF 
RECIPIENTS 
WITH ERROR 



PERCENT OF 
DOLLARS IN 
ERROR 



MEAN ERROR 
PER RECIPIENT 
WITH ERROR (S) 



Mother's Expected 1988 Income From 
Work 

PiKoU' Expected 1988 Other Taxable 
Income 

Pareau' Expected 1988 Untaxed 
Income and Beneflti 

PareeU' 1987 Social Security 
Benefits 

ParenU' Businea and Farm 
Value 

Pareau' 1987 Education Expenses 

ParenU' Expected 1988 Other 
Taxable Income 

Pareou' 1987 Itemized Deductions 

ParenU' Other Real Estate/ 
Inveatroet Debt 

ParenU* Diqilaced Horoemaker 
Status 

ParenU' 1987 Exemptions 

ParenU' 1987 ChUdren With 
Education Expenses 

ParenU' Marital Status 



0.S 

0.3 

0.3 

0.2 

0.1 

0.1 
0.1 

0.1 
0.1 



OA 



0.1 



0.1 



0.1 



662 

828 

2077 

448 

616 

186 
1023 

194 
543 

36 



217 
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STUDENT MARGINAL ERROR-CAMPUS-BASED NEED ERROR 
Rtnked by FnqiMocy uni DolUr ImpMt 
DEPENDENT STUDENTS ONLY 



APPUCATION ITEM 



PERCENT OF 
RECIPIENTS 
WITH ERROR 



PERCENT OF 
DOLLARS IN 
ERROR 



MEAN ERROR 
PER RECIPIENT 
WITH ERROR ($) 



PanoU' 1987 AFDC of ADC 



500 



ParaoU' 1987 Filing Sutiu 



Ptnou' State of Legal Retidence 



* Lew than 0.05% 
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STUDENT MARGINAL ERROR-^AMPUS-BASED NEED ERROR 
RMkDd by Ficquaicy and Dollar Impact 
INDEPENDENT STUDENTS ONLY 



APPUCATION ITEM 



PERCENT OF 
RECIPIENTS 
WTTH ERROR 



PERCENT OF 
DOLLARS IN 
ERROR 



MEAN ERROR 
PER RECIPIENT 
WITH ERROR ($) 



Studcat's Number in College 

Student's Houaebold Size 

Student's (& spouse's) 1987 Medical 
and Dental Expensea 

Student's (ft spouse's) 1987 
Education Expenses 

Student's (& qiouae's) 1987 
Children with Education Expenses 

Student's Veterans Contributory 
Benefiu Per Month 

Student's Veterans Contributory 
BeneHu Number of Months 



3.5 
3.4 

1.0 

0.2 



0.2 
1.2 



261 
1850 
171 



♦ Less than 0.05% _ 

233 
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ALL STUDENTS (DEPENDENT AND INDEPENDENT) 



Exhibit A-6 
P«telof3 



APPUCATION ITEM 



PERCENT OF 
RECIPIENTS 
WITH ERROR 



PERCENT OF 
DOLLARS IN 
ERROR 



MEAN ERROR 
PER RECIPIENT 
WITH ERROR ($) 



Student's (and Spouw's) Caah, 
Saving!, Cbscking 

Student's (and Spouae's) Income 

Student's 1987 Work Income 

Student's (and Spouae's) 1987 Total 
Untaxed Income and BeneHu 

Student's (ft apouse's) 1987 Other 
Untaxed Income and Benefits 

Student's (and Spouae's) 1987 Taxes 
Paid 

Student'e (and Spouae's) Real 
Eitate/Inveatroeat Value 

Year in College 

Student's (and Spouae's) Home Value 

Spouse's 1987 Work Income 

Student's (and Spouse's) Social 
Seauity Beoefito 

Ward of the Court 

Legal Dependents other than Spouse 

Student's Expected 1988 Income 
From Work 



4.3 

3.8 
3.6 
2.4 

2.2 

1.9 

1.2 

0.8 
0.6 
0.6 
0.4 

0.3 
0.3 
0.3 



0.3 

0.5 
0.3 
0.5 

0.4 

0.1 
0.2 



0.4 



0.1 



0.2 



299 

645 
335 
843 

898 

225 

659 

192 
3318 
329 
739 

2412 
830 
I7t 



Student's Expected 1988 Other Income 



0.3 



97 



* LcM than 0.05% 
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Exhibit A-6 
Page 2 of 3 



APPUCATION ITEM 



PERCENT OF 
RECIPIENTS 
WITH ERROR 



PERCENT OF 
DOLLARS IN 
ERROR 



MEAN ERROR 
PER RECIPIENT 
WITH ERROR (S) 



Student's (and Spouae's) Home Debt 

Student's Expected GI Bill-Ddlara 

Spouse's Expected 1988 Income From 
Work 

Student Dislocated Worker Status 

Student's Date of Birth 

Student's Displaced Homemaker Status 

Student's Expected GI Bill-Months 

Student's (& spouse's) 1987 AFDC 
or ADC 

1985 Total Resources Of $4,000 or 
More 

Student's State of Legal Residence 

Student's Marital Status 

Student's 1987 AFDC or AFD 

1987 Total Resources Of S4,000 or 
More 

U.S. Citizen 

Candidate for a Bachdor't Degree 



0.2 
0.2 

0.2 

0.2 
0.1 
0.1 



661 
1275 
377 

462 
94 

489 



140 



* LeM than 0.05% 
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APPUCATION ITEM 



PERCENT OF 
RECIPIENTS 
WITH ERROR 



PERCENT OF 
DOLLARS IN 
ERROR 



MEAN ERROR 
PER RECIPIENT 
WITH ERROR ($) 



StudcQi'i (A •pouae'i) Ocber Real 
Ealate Inveatmeot Debl 

Studeot'i (& apottae*i) 1987 
Exempdooa 

Student's {Sl ipouae's) Filing Status 

Received Fedetil Student Aid 1987*88 

Claimed as Exemptions on Parents 

1987 Income Tax Return 

Claimed as Exemptioos on Parents 
1986 Income Tax Return 

Claimed as Exemptioos on Parents 

1988 Income Tax Return 

1986 Total Reaourcea Of $4,000 or 
More 

Student's (& qwuse's) Itemized 
Deductions 

Student's (* spouse's) Business 
And Farm Value 

Studmt's (& spouse's) Business 
And Farm Debt 

Veteran of the Armed Forces 

Student's 1987 Child Support Received 

Student's Expected 1988 Untaxed Income 



140 
155 



1407 



155 



1407 
195 
155 



^ Leu than 0.05% 



23B 
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Page 1 of 2 



STUDENT MARGINAL ERROR-STAFFORD OVERCERTDPICATION ERRCil 
IRAoked by Frequeocy and Dollar Error 
DEPENDENT STUDENTS ONLY 



APPLICATION ITEM 



PERCENT OF 
RECIPIENTS 
WITH ERROR 



PERCENT OF 
DOLLARS IN 
ERROR 



MEAN ERROR 
PER RECIPIENT 
WITH ERROR (S) 



Panou' Houaabold Size 

Parents' Home Value 

PareoU' 1987 Other Untaxed 
Income and Beoefiu 

Parent' Number in College 

Parent's Social Security Benefits 

ParenU' Other Real/Investment Value 

ParenU' 1987 Income 

Parentt' Home Debt 

Father's 1987 Income From Woik 

Parents' Cash, Checking, Savings 

PaienU' 1987 Taxes Paid 

Mother's 1987 Income From Work 

Age of Older Parent 

ParenU' 1987 Child Support Received 

Parents' Busineu and Farm Value 

Father's Expected 1988 Other Taxable 
Income 



5.4 

4.0 
3.7 

2.6 
2.6 
2.4 
1.6 
1.2 
1.2 
1.1 
1.0 
0.7 
0.6 
0.4 
0.4 
0.3 



1.4 

1.0 
0.5 

1.2 
0.7 
0.7 
..2 
0.2 
0.1 

0.3 
0.1 
* 

0.2 
O.l 



600 
541 
313 

1063 
613 
682 

1675 
294 
264 
147 
741 
205 
99 

1400 
444 
548 



* Lou than 0.05% 
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STUDENT MARGINAL ERROR- STAFFORD OVERCERTIFICATION ERROR 
Ranked by Frequency and Dollar Error 
DEPENDENT STUDENTS ONLY 



APPUCATION ITEM 



PERCENT OF 
RECIPIENTS 
WITH ERROR 



PERCENT OF 
DOLLARS IN 
ERROR 



MEAN ERROR 
PER RECIPIENT 
WITH ERROR (S) 



Pareou' 1987 Medical and Dental 
Ejqwoaea 

PaitoU' Expected 1988 Other Taxable 
Income 

Pareota' Diqdaced Homeroaker Status 

Molher'a Expected 1988 Income From 
Work 

Parenu' Marital Statua 

Paienta' Expected 1988 Untaxed 
Income and Beoeltta 

Pareota' State of Legal Residence 

Parents' 1987 Cluldrea With 
Educational Expmat 

Pareoto' 1987 Filing Statua 

Parents' 1987 Exemptions 

Parenta' 1987 Education Expenses 

Parenu' Other Real Estate / 
Inveatmeot Debt 

PaienU' 1987 AFDC or ADC 

Parenu' 1987 Itemized Deductions 



0.3 

0.2 

0.1 
0.1 



0.1 



376 

1563 

406 
416 



* Lea than 0.05% 



238 



A-17 



Exhtbk A-8 



STUDENT MARGINAL ERROR--STAFFORD OVERCERTinCATION ERROR 
Rtakod by FrajuMcy tad DoUar Error 
INDEPENDENT STUDENTS ONLY 



PERCENT OF PERCENT OF MEAN ERROR 
RECIPIENTS DOLLARS IN PER RECIPIENT 

APPUCATION ITEM WITH ERROR ERROR WITH ERROR (S) 



Studeat'i HotttetMld Size 0.7 1.0 4462 

Student*! Numlm In College 0.2 

Student's (Ac fpouae*i) 1987 ^ 
Education Expcntea 

Student*! (& tpouae's) 1987 Medical ^ 
and Dental Expwaea 

Student*! (ft tpouae*!) 1987 ^ 
Children with Education Expenies 

Student*! Veteraaa Contributory ^ 
Benefit! Per Month 

Student*! Veterana Contributory ^ 
BeneffiU Number of Month! 



152 



♦ Let! than 0.05% 
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Exhibit A-9 
Page 1 of 3 

STUDENT MARGINAL ERROR-STAFFORD OVERCERTIRCATION ERROR 
Ranked by Fraqococy tod Dollar Error 
ALL STUDENTS (DEPENDENT AND INDEPENDENT) 



PERCENT OF PERCENT OF MEAN ERROR 
RECIPIENTS DOLLARS IN PER RECIPIENT 

APPUCATION ITEM WITH ERROR ERROR WITH ERROR (S) 



Studeot't (St ipouae's) 1987 Income 

Student's (& ipouae's) 1987 Other 
Untaxed Income and Beoefiti 

Student's 1987 Income From Work 

Student's 1987 Total Untaxed Income 
and Benefits 

Year in College 

Student's (& spouse's) Cash, 
Savings, Cbeeldng 

Student's (& spouse's) 1987 Taxes 

Student's (St spouse's) 1987 Social 
Security Beaenta 

Student's (St spouse's) Home Value 

Student's (St qwuse's) Other Real 
Estate/Investment Value 

Spouse's 1987 Income From Work 

Student's (St spouse's) Home Debt 

Student's (St spouse's) Expected 1988 
Untaxed Income and Beneflts 

Student's Expected GI Bill-DoUars 

Student's (St spouse's) 1987 Child 
Support Received 



2.2 
1.5 

1.4 
1.3 

1.0 
1.0 

1.0 
1.0 

0.4 
0.4 

0.2 
0.1 
0.1 

0.1 



1.0 
0.4 

0.4 
0.6 

0.6 
0.1 

0.1 



0.4 
0.2 



1234 
748 

747 
1159 

1538 
170 

184 
742 

2382 
1125 

186 
136 
136 

100 
156 



* Less than 0.05% 
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Exbibil A-9 
Page 2 of 3 



STUDENT MARGINAL ERROR-STAFFORD OVERCERTinCATION ERROR 
Raolood by Frequflocy and Dollar Error 
ALL STUDENTS (DEPENDENT AND INDEPENDENT) 



PERCENT OF PERCENT OF MEAN ERROR 
RECIPIENTS DOLLARS IN PER RECIPIENT 

APPUCATION ITEM WITH ERROR ERROR WITH ERROR ($) 



1987 Total ReaouiCM Of $4,000 or More 

Claimed ai Exemptions oo Parenti 

1986 Income Tax Return 

Student's (& spouse's) Other Real 
Estate/Iflvcstmeot Debt 

Claimed aa Exemptiocis oo Parents 

1987 Income Tax Return 

U.S. Citizflo 

Student's (& spouse's) Dislocated 
Worker Statue 

U.S. Armed Forces Vcieran 

Student's (& spouse's) 1987 AFDC or ADC 

Student's (& spouse's) Displaced 
Horoeroaker Status 

Student's Expected 01 Bill-Months 

Student's Marital Status 

Spouse's Expected 1988 Income 

Student's Date of Birth 

1986 Total Resources Of $4,000 or More 



336 



^ Leu than 0.03% 
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Exhibit A-9 
P«gi> 3 of 3 



STUDENT MARGINAL ERROR-STAFFORD OVERCERTIFICATION ERROR 
Ranked by Freqimcy tod DoUir Error 
ALL STUDENTS (DEPENDENT AND INDEPENDENT) 



PERCENT OF PERCENT OF MEAN ERROR 

RECIPIENTS DOLLARS IN PER RECIPIENT 

APPLICATION ITEM WITH ERROR ERROR WITH ERROR ($) 

Smdeot'i (& ipouio'i) 1987 ^ # # 
ExciQptioQS 

1985 TotilRiMOurctt Of $4,000 or More ^ # ^ 

Legal Dependent Other Than Spouse # ♦ ^ 

Student*! Expected Income From # # # 
Work 



Student's (& qmiae*!) Filing Status 

Student's (& spouse's) Itemized 
Deductions 



Claimed as Exemptions on Parenu 
1988 Tax Return 



Student's (& qxMise's) Business/Farm Value 



Ward of Court 



Candidate For Bachelor's Degree 



Received Federal Student Aid 1987-88 



Student's (& spouse') ) Business/Farm Debt 



Student's (& spouse's) Expected 1988 
Taxable Income 



693 



Student's State of Legal Residence 



^ Leu than 0.05% 
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APPENDIX B - VARIABLES ASSOCIATED WITH ERROR 



This appendix presents the results of the contingency table analyses conducted as the 
first step in the profile error analysis presented in chapter IV. Contingency table analysis 
is used to determine associations between variables. For this study, contingency tables 
were used to examine the relationship between the incidence of error (with a $50 
tolerance) and a variable. 

As the first step in this analysis, categories were foimed for all of the variables used in 
the analysis. For categorical variables (e.g., yes/no questions, institution type and 
control, etc.), categories were formed through a ranking process. Each value of a 
variable was assigned a score equal to the percentile represented by that value (e.g., the 
median value for family income was replaced by a score of 50). These scores were all 
between 0 and 100. The scores were the partitioned into the following categories: 

• 10 or less, 

• 11-25, 

• 26 • 50, 

• 51 - 75, 

• 76 - 90, or 

• 91 - 100. 

After all the variables were categorized using one of the above methods, contingency 
tables were formed that compared the categories of each variable to the presence of 
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error (using a $50 tolerance). 

Each variable tested for association with student error was tested on the relevant 
population of students (i.e., dependent, independent, or all). Parent data (e.g., parent's 
adjusted gross income and the number in the parent's household), which are reported 
only by dependent students, were tested on a subset of the database containing only 
dependent students. Similarly, variables that are reported only by independent students 
(e.g., student's household size) were tested using a subset of the database containing only 
independent students. Variables reported by both independent and dependent students 
(e.g., student's adjusted gross income) were tested using the full database of all students. 
Each variable tested for association with institutional error was tested on the entire 
population of students. 

The remainder of this appendix consists of exhibits presenting the results of the 
contingency table analysis. The exhibits presented include: 

• Variables Tested for Association with Student Error - a listing of all of the 
variables tested for association with student error, whether the association 
was significant, and the category of the variable with the highest probability 
for error. 

• Variables Associated With Student Erro r/Pell Grant Program - a listing of 
all variables that were significantly associated with error at the 0.05 
significance level. 

• Variables Associated With Student Err or /Campus-Based Program a 
listing of all variables that were significantly associated with error at the 
0.05 significance level. 
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Variables Associated With Student Errnr/Stafford T^an Program - a 
listing of all variables that were significantly associated with error at the 
0.05 significance level. 

Variables Tested for Association with Institutional Error - a listing of all 
of the variables tested for association with institution error, whether the 
association was significant, and the category of the variable with the highest 
probability for error. 

Variables Associated With Institutional Error /Pell Grant Program -- a 
listing of all variables that were significantly associated with error at the 
0.05 significance level. 

Variables Associated With Institutional Error/Campus-Based Program - a 
listing of all variables that were significantly associated with error at the 
0.05 significance level. 

Variables Associated With Instimtional Error /Stafford Loan Program -- a 
listing of all variables that were significantly associated with enor at the 
0.05 significance level. 
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EXHIBIT B-.1 

Pago 1 of 4 



VARIABLBS TESTED FOR ASSOCIXTIQW WITH S' 



ERROR 



Studant Characf rl«tle« 



Significant^ 
Association 

With Error 



Catagory with lighast 
ProbabilitT of Error 



AFOC Racaivad 
Aga 

AGI 

Application Data 
Aaaata 

Availability of Information Used to Fill 
Out Application 

Citizonahip 

Oapandancy Status 

Difficulty in Undarstanding Applicatir i 
Elanantary/Sacondary Tuition 

Family Anaats 
Family Incoma 
Grada Point Avaraga 
Homa Valua 

Houaahold Siza 

Lagal Dapandanta othar Than Spousa 

Marital Status 
Nadical/Li'ntal Expansas 



Yaa 
Yea 
Yaa 

No 
No 

Yes 

No 
No 
Yaa 

Yas 

Yea 
Yaa 

No 
Yes 

No 
Yes 

Yes 
Yes 



Without AFDC 
Under 32 year a 
Higheat incoma 

N/A 

N/A 

Don't know 

N/A 

N/A 

Don't know 

Have tuition 
expanse 

Highest asseta 
Higheat incoma 

N/A 

Higheat homa 
value 

N/A 

Have no legal 
dependents 

Single 

Have medical/dental 
expenses 



At 0.05 significance level. 
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EXHIBIT 

Pag« 2 of 4 



VARIABLES TESTED FOR ASSOCIATION WITH STUDENT ERROR 



StttdMt Chayaef rl«tlts« 

Not Assets 

Net Family Assets 

Net Home Value 

Number in College 

Previous Bachelor's Degree 

Received Campus-^Baeed Aid 

Received Help in Filing Financial Aid 
Application 

Sex 

Tax Form Filed 
Total Expenses 
Types of Aid Received 

Untaxed Income 

Veterans status 

Ward of the Court/Parents 
Deceased 

Year in College 



Significant^ 
Association 

With Error 

NO 
Yes 

No 

No 
Yes 
Yes 

Yes 
Yes 
Yes 
Yes 
Yes 

Yes 
Yes 

No 
Yes 



Category witli Highest 
Probability of Error 



N/A 

Highest Assets 
N/A 
N/A 

No previous degree 

Received CB award 

Someone else filled 
out application 

Hale 

Estimated 1040A 

Highest expenses 

Highest number of 
awards 

Least untaxed income 
Non veterans 



N/A 

Undergraduate 



At 0«05 significance level. 
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P«9« 3 of 4 



VARIABLES TESTED FOR ASSOCIATION WITH 5STDDENT ERROR 



Parant Charaetaristics for 
Daoandent Studanta 

AFDC Reeoivad 
AGI 

Aaaata 

Elamentary/Sacondary Tuition 
Homa Valua 
Houaahold Siza 
Marital Statua 
Madical/Dantal Expanaea 
Nat Homa Valua 

Nat Aaaata 
Nuinbar in Collaga 
Tax Form Filed 
Total Incoma 
Total Expanaaa 
Untaxad Incoma 



Significant' 
Aaaociation 

With Error 

Yea 
Yea 
Yea 

No 
Yaw 

No 

No 
Yea 
Yea 

Yea 
No 
Yea 
Yea 

Yea 
Yea 



Category with Bigheat 
Probability of Error 



Without AFDC 
Higheat income 
Higheat aaaata 
N/A 

Higheat home value 
N/A 
N/A 

Higi.sat expenaea 

Higheat net home 
value 

Higheat net aaaeta 

N/A 

Eatimated 1040 
Higheat income 
Higheat expenaea 

Loweat untaxed 

income 



' At 0.05 significance level. 
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EXHIBIT F -1 
Pig« 4 ol; 4 



VARIABLES TESTED FOR ASSOCIATION WITH STUDENT ERROR 



inatltutional Chara ef giitlc« 



Significant^ 
Aaaeciation 



Catagory with Bighaat 
Probability of grror 



Control 

How infornation la providad to atudants 
on inportanca of providing accurate data 
and conaaquancaa of miaraporting 

Itama varifiad for inatitution 
varif ication 

Nead Analyaia servicar uaad for 
largest number of students 

Policy on tax form collection 

Region 

Type 

Use of automation in addition to reports 
from out aide sourcea 

When information ia provided to students 
on importance of providing accurate data 
and consequences of misreporting 



Yes 
Yes 

Yes 

Yes 
No 
No 

Yes 

No 
No 



Proprietary 

Information ia not 
provided 



Uaa IV itema plua 
other itema 

Processor other than 
Federal 

N/A 

N/A 

4 yr. Baccalaureate 

N/A 



N/A 



* At 0.05 significance level. 
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EXHIBIT B-2 
Page 1 of 2 

VARIABLES ASSOCIATED WITH STUDENT ERROR* 
PELL GRANT PROGRAM 



Studant Gharaef riatlcs 



Variable Probability 

AGI 0.000 

Untaxed Income 0.000 

Total Income 0.000 

Family Income 0.000 

Total Expenses 0.023 

Elementary/ Secondary Tuition 0.037 

Net Assets 0.040 

Family Assets 0.000 

Net Family Assets 0.000 

AFDC Received 0.000 

Household Size 0.017 

Age 0 . 000 

Citizenship 0.019 

Legal Dependents Other Than Spouse 0.000 

Received Help in Filing Financial Aid Application 0.000 

Marital Status 0.001 



*At .05 tignificano* l*v«l 
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EXHIBIT B-2 

Page 2 of 2 



VMIZABLES ASSOCIATED WITH STUDENT ERROR* 
PELL GRANT PROGRAM 



Parent Character iatiea for Dependent Students 



Variable 



Probability 



AGI 

Untaxed Income 

Total Income 

Total Expenses 

Medical /Dental Expenses 

Home Value 

Net Home Value 

Net Assets 

Assets 

AFDC Received 
Tax Form Filed 



0.000 
0.006 
0.000 
0.000 
0.032 
0.000 
0.000 
0.000 
0.000 
0.000 
0.000 



*At .05 tignificanot l«v«l 
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Page 1 of 2 



VMIIABLES ASSOCIATED WITH STUDENT ERROR* 
CAKPUS-BASED AXD 



Studant Gharactariatics 



Variable 


Probabilitv 


A6I 




0.000 


Untaxed Income 




0.000 


Total Income 




0.000 


Family Income 




0.000 


Total Expenses 




O.Olf 


Medical/Dental Expenses 




0.002 






0.000 


Net Family Assets 




0.000 


AFDC Received 




0.000 


Age 




0.000 


Year In College 




0.004 


Previous Bachelor's Degree 




0.001 


Legal Dependents Other Than Spouse 




0.000 


Sex 




0.024 


Received Help in Filing Financial Aid Application 




0.000 


Availability of Information Used to Fill Out Applicat 


ion 


0.019 


Marital Status 




0.000 


Tax Form Filed 




0.042 



^Al .03 significano* Itvcl 
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EXHIBIT B-3 

Page 2 of 2 



VARIABLES ASSOCIATED WITH STUDENT ERROR* 
CAMPUS-BASED AID 



Parant CharaetTiatles for Dependant Students 

AGI 0.000 

Total Income 0.000 

Total Expenses 0.000 

Medical/Dental Expenses 0.004 

Home Value 0.038 

Net Home Value 0.042 

AFDC Received 0.000 

Tax Form Filed 0.001 



*Ai .KS cignlficanM Itvtl 
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EXHIBIT B-4 



VARIABLES ASSOCIATED WITH STUDENT ERROR* 
STAFFORD LOAM PROGRAM 

8tud«nt Characf riatlea 



Variable PfQ^UitY 

AGI 0.000 

Total Income 0.000 

Family Income 0.000 

Total Expenses 0.044 

Assets 0.033 

Family Assets 0.000 

Net Family Assets O.OOO 

Age 0.000 

Year In College 0.018 

Veterans Status 0.009 

Legal Dependents Other Than Spouse 0.000 

Received Help in Filing Financial Aid Application 0.035 

Marital Status 0.000 

Parent Characteriatics for Dependent Students 

AGI 0.016 

Total Income 0.006 

Total Expenses 0.007 

Home Value 0.015 

Net Home Value 0.011 

Net Assets 0.002 

Assets 0.008 



*At .05 tigniflcanc* levtl 
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EXHIBIT B"S 
Paga 1 of 2 



VMIXABLBS TBSZEO FOR AS80CZAXZ0M WZIH ZNSSZTUTZOKAL ERROR 



In»tittttlonal Characteri«tleE 
Academic Calandar 

Naada Analysis Servicar Uaad for Host 
Studanta 

Tit la ZV Programa Participated In 

Use of Automation In Addition to Reports 
From Outside Sources 

Do Institutional Verification 

Policy on Tax Form Collection 

Itama Verified During Institutional 
Verification 

Method to Ra-check Institution Data 
and Calculations 

Method to Re-check Files for 
Documentation 

When Information la Given to Students 
Regarding Importance of Accurate Data 
and Penalties for Error 

How Information Is Given to Students 
Regarding Importance of Accurate Data 
and Penalties for Error 

Region of Country 

Institution Type 



Significant 
Aasociation 

With Error 

No 



No 
No 

No 
No 
Yes 

No 

No 

No 

No 

Yes 
Yes 
Yes 



1 



Category with 
Highest Error Rate 

N/A 

N/A 
N/A 

N/A 
N/A 

From all students 
N/A 
N/A 
N/A 

N/A 

Information not given 

No Pattern 

Non-Baccalaureate 
Program 



Institution Control 



Yes 



Proprietary 



At 0.05 significance level. 
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EXHIBIT B-5 



Pag* 2 of 2 



VMIIABLBS TB8TED FOR ASSOCIAZXOM NIXB IMSTITDTIONAi;. BRXOR 

.2 



Inatlttttlonal Charaet«ri«tle« 



Significant' 
AsBoeiation Catagory with 



With Error 



HiqhiiJt Error Raf_ 



Nufflbar of Titla IV Awards 

Nunbar of Full-Tima Equivalant Staff 

Titla IV Awards Par Full-Tima 
Equivalant Staff 

Titla IV Awards Par Full-Time Equivalent 
Professional Staff 



Yes 

NO 

Yes 

No 



No pattern 
N/A 

No pattern 

N/A 



Student characteristics 

Types of Aid Received by the Student 

Indicator That Student Received a 
Pell Grant 

Indicator That Student Received a 
Caropus-Based Award 

Indicator That Student Received a 
Stafford Loan 

Student Dependency Status 



Yes 
No 
Yes 

Yes 

No 



Number of Awards 



N/A 



Received C-B Award 



Received SL Award 



N/A 



^ At 0.05 significance level. 
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EXHIBIT B-6 



VARIABLES ASSOCIATED WITH IMSTITUTIONAL ERROR 

P«ll Orant Program 



inatlttttlonal Charaetarlatloa 

Needs Analysis Servicer Used for Most Students 

Method to Re-check Institution Data 
and Calculations 

How Information Is Given to Students Regarding 
Importance of Accurate Data and Penalties 
for Error 

Region of Country 

Institution Type 

Institution Control 

Title IV Awards Per Full-Tlme Equivalent Staff 



0.006 
0.000 

0.000 
0.039 
0.005 
0.033 
0.038 



Student Characteriatiea 

Types of Aid Received by the Student 



Probabilitv 
0.000 
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EXHIBIT B-7 



VIkRIABLES ASSOCIATED WITH INSTITUTIONAL EKROR 

caapui-Bastd Program 



inatlttttlonal charaof rlatlcs 

Policy on Tax Form Collection 

How Information Is Given to Students Regarding 
Importance of Accurate Data and Penalties 
for Error 

Region of Country 

Institution Type 

Title IV Awards Per Full-Tlme Equivalent 
Professional staff 



Probability 
0.048 

0.001 
0.006 
0.000 

0.021 



Student Ch araete riatica 
Types of Aid Received by the Student 
Indicator That Student Received a Pell 
Indicator That student Received a Stafford Loan 



Probability 
0.000 
0.002 
0.002 
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EXHIBIT B~8 



VARIABLES ASSOCIATED WITH INSTITUTIONAL ERROR 
Stafford Loan Prograa 



instlttttlonal Charaef riatiea 

Needs Analysis Servicer Used for Most Students 

Method to Re-check Institution Data 
and Calculations 

How Information Is Given to Students Regarding 
Importance of Accurate Data and Penalties 
for Error 

Region of Country 

Institution Type 

Institution control 

Number of Title IV Awards 



Probability 
0.022 

0.013 

0.000 
0.039 
0.042 
0.019 
0.040 



Student Gharaeteriatiea 

Types of Aid Received by the Student 

Indicator That Student Received a Pell 

Indicator That Student Received a Campus-Based 
Award 

Dependency status 



Probability 
0.000 
0.000 

0.036 
0.000 
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APPENDIX C - FEDERAL GOVERNMENl i:OSTS PER DOLLAR OF STAFFORD 

LOAN 



This appendix presents a reproduction of the worksheets used by ED to determine the 
cost of each dollar that is loaned through the Stafford Loan program, lias worksheet 
was provided for use in estimating the cost to the Federal govemmtnt for Stafford 
Overaward errors. 
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EXHTBTT r.1 



LIFE OF LOAN PER DOULAR COSTS OF TYPICAL" STAFFORD LOAN 



OBUGATIONS 




INTEREST SUBSIDIES 




Interest Benefits 


0.240 


Special Allowance 




Subtotal 


0.438 


OTHER SUBSIDIES 




Administrative Cost Allowance 


0.010 


DEFAULTS & RELATED COSTS 




Defaults 


0.140 


Death and Disability 


0.004 


DanKrupicy 




Subtotal 


0.150 . 


TOTAL OBLIGATIONS 


0.598 


RECEIPTS 




Loan Obligation Fee 


0.050 


Collections 


0.035 


TOTAL RECEIPTS 


0.085 


NET COST 


0.513 



ASSUMPTIONS/NOTES: 

(1) T-bill rates are ftMUBtted M 0.078. 

(2) In school life of loan « 3.0 yrs (l.S^^.S grace period). 
(3i Repayment period » 7 years. 

(4) Interest rate»0.08. 

(5) For default, death and disability, and bankruptcy rates, current experience is assumed. 

(6) Administrative Cost Allowance is assumed to be paid. 

This data represenu estimatetf from the Department of Education as of 1 1/21/89. 



ERIC 



APPENDIX D - EXAMPLES OF CONTINGENCY TABLE ANALYSIS 

This appendix presents two examples of the contingency table analysis used during early 
stages of the error profile analysis. The two exhibits are as follows: 

• Example of Contingency Table Analysis for Institution Error 

• Example of Contingency Table Analysis for Student Error 
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EXHiBrr D-i 



EXAMPLE OF CONTINGENCY TABLE ANALYSIS FOR INSTITUTION ERROR 



Which of tht following statements best expresses your policy regarding 
submission of Federal Income tax returns or statements of non^flUng? 



I FUG VALUE 

I net 1 tut 1 on Response 

Error 



FREQUENCY 
COL PCT 


All student A 
and partntt 

mutt subnit 
<1) 


All ED seltcted 
students and 
parents subnit 
(2) 


All verified 
students and 
parents subnit 

(4) 


Other 


TOTAL 


NO 


786 
77.29 


268 
79.29 


899 
82.70 


75 
84.27 


2028 

► 


YES 


231 
22.71 


70 
20.71 


188 
17.30 


U 
15.73 


503 


TOTAL 


1017 


338 


1087 


89 


► 

2531 



STATISTICS FOR TABLE OF I FLAG BY VALUE 



STATISTIC 


OF 


VALUE 


PROS 


CHI-SQUARE 


3 


10.798 


0.013 


LIKELIHOOD RATIO CHI-SQUARE 


3 


10.840 


0.013 


NANTEL-HAENSZEL CHI -SQUARE 


1 


1.525 


0.217 


PHI 




0.065 




CONTINGENCY COEFFICIENT 




0.065 




CRAMER'S V 




0.065 





SAMPLE SIZE * 2531 
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EXHIBIT D>2 



EXAMPLE OF CONTINGENCY TABLE ANALYSIS FOR STUDENT ERROR 



SFLAG VALUE 

Studtnt Student's Adjusted Gross Income 
Error 



FREQUENCY 


$0 ond 


$1 ■ 


S601 - 


$2,690 • 


$6,586 • 


$15,554 


COL PCT 


Under 


$600 


S2,689 


S6.585 


$15,553 


■nd Over 




10 


25 


50 


75 


90 


100 


No 


332 


1 


369 


431 


294 


181 




68.03 


55.78 


58.57 


68.09 


77.57 


71.26 



Yet I 156 I 65 I 261 I 202 I 85 I 73 I 842 

I 31.97 I 44.22 | 41.43 | 31.91 | 22.43 | 28.74 | 

.... > > — . — > 

TOTAL 488 147 630 633 379 254 2531 



STATISTICS FOR TABLE OF SFLAG BY VALUE 



STATISTIC 


OF 


VALUE 


PROB 


CHI*SQUARE 


5 


50.142 


0.000 


LIKELIHOOD RATIO CHI -SQUARE 


5 


50.632 


0.000 


MANTEL 'HAENSZEL CHI -SQUARE 


1 


12.802 


0.000 


PHI 




0.141 




CONTINGENCY COEFFICIENT 




0.139 




CRAMER'S V 




0.141 





SAMPLE SIZE « 2531 
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